Predicting outcomes of digital therapeutics and other interventions in clinical research

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for using machine learning to generate precision predictions of readiness. In some implementations, a database is accessed to obtain status data that indicates activities or attributes of a subject. A set of feature scores is derived from the status data for the subject, the set of feature scores including values indicative of attributes or activities of the subject. The set of feature scores to one or more models that have been configured to predict readiness of subjects to satisfy one or more readiness criteria. The one or models can be models configured using machine learning training. Based on processing performed using the one or more machine learning models and the set of feature scores, a prediction regarding the subject&#39;s ability to achieve readiness to satisfy the one or more readiness criteria is generated.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under SBIR grantHHSN261201700003C awarded by the National Cancer Institute. Thegovernment has certain rights in the invention.

BACKGROUND

Some computer systems make estimates or predictions about the expectedduration of a process, such as an application installation or filetransfer. Some systems may estimate when a subject will reach a certainstate, such as a state of operability or completion of a task. Inpractice, however, the accuracy of these estimates is often low, andperformance varies widely from one task to the next, even for estimatesregarding short, simple tasks.

SUMMARY

In some implementations, a computer system is configured to use machinelearning techniques to predict when and if a subject will reach adesired level of readiness, e.g., a desired state or degree of abilityneeded to perform a task or achieve an objective. In general, thedesired level of readiness may involve the subject acquiring acapability to perform certain actions or functions. The desired level ofreadiness may represent the completion of a specific task, and thusreadiness to perform another activity. As another example, separate fromany individual task, the desired level of readiness may refer toacquiring the capability to perform a task, even if one has not beenattempted or initiated. In addition, or as an alternative, the desiredlevel of readiness may involve the subject reaching a certain physicalstate or configuration (e.g., some predetermined status or condition ofthe subject).

The computer system can use various techniques, including machinelearning techniques, to generate a variety of predictions regardingreadiness of a subject. For example, the computer system can use amachine learning model to predict when a subject will reach a certainlevel of capability to be able to perform a task or will achieve anotherresult (e.g., to complete a task in progress). Similarly, the computersystem can use a machine learning model to predict whether a subjectwill reach the level of capability under certain conditions, such aswithin a defined time period, within an amount of time, or given acertain amount of resources. As another example, the computer system canuse a machine learning model to predict whether a subject will ever beable to achieve the level of capability given various factors, which mayinclude resources allocated to the subject and/or activities (e.g.,current or planned) designed to enhance the capabilities of the subject.As another example, the computer system can use a machine learning modelto predict how long a subject will maintain readiness, with or withoutactivities planned to assist in maintaining the desired capabilities.The computer system can also use machine learning techniques to provideother outputs, such as scores that indicate a predicted measure ofreadiness (e.g., level of capability) of the subject at the current timeor at a future time. As the system collects information and makespredictions, it can update its models to learn from the additionalexamples and feedback received.

In some implementations, the computer system can use machine learningtechniques to evaluate and recommend actions for enhancing the readinessof a subject. This can include, for example, recommending actions totake to improve readiness, indicating the impact of specific actions onreadiness of the subject, identifying factors that support or hinderreadiness of the subject, evaluating different options for buildingcapabilities of the subject, predicting the progression or trend ofreadiness for the subject under different conditions, and so on. Thus,besides predicting when and if a subject will achieve readiness, thecomputer system can assess and recommend options to improve the processof achieving readiness, e.g., to allow the subject to reach readinessfaster or to reach a higher level of capability.

The system is capable of providing predictions regarding readiness, aswell as recommendations for improving readiness, for many types ofsubjects and for many different types of readiness. For example, thesubject may be a computer system, a phone, a mobile device, software, avehicle, a robot, an item being fabricated, a factory, an organization,a person, a team of people, and so on.

In general, the system can be applied in many fields, in order topredict when a subject will achieve a desired state of readiness orcapability. The desired state may be defined in terms of physicalstructure or composition, such as a physical arrangement of a mechanicalsystem, a physiological and/or psychological state of a person, etc. Inaddition, or as an alternative, the desired state may be defined interms of functional ability, such as the ability of a device to performa task, a result of generating a file or data output, a capability of aperson to perform a task, etc.

Readiness can be different for different subjects, and can refer to asubject achieving a physical state and/or functional state. Readinessmay be defined and measured in different ways, for example, with respectto physical attributes of a subject, functional capabilities of asubject, levels of experience or prior activities performed by thesubject, and so on. In the example of a computer system as the subject,readiness may represent the computer system achieving a certain level ofprocessing capability (e.g., acquiring the capability to be able toperform tasks of a predetermined type, or to perform the tasks with adesired level of speed or efficiency). As another example of a person asthe subject, readiness may represent the person reaching a certain levelof health or physical fitness, obtaining a level of proficiency at askill, etc. Readiness may span many different dimensions, such as thecase of readiness to take on a particular job role, where readiness mayhave components of level of experience, physical capabilities, knowledgelevel, skills acquired, and so on.

For each of various applications, one or more models can be generatedbased on example data sets. The data sets can provide examplesdescribing the process by which other subjects achieved readiness. Forexample, the data sets may describe the progression of variousindividual subjects toward achieving a capability or state. The datasets can show examples that were successful in achieving readiness, aswell as examples that were not successful. A computer system can use thepatterns and characteristics of the various examples to generate modelsthat can be used to predict the readiness of other subjects based onvarious factors. Examples of factors that can be used for prediction arecurrent and former attributes of the subject, current and formercapabilities of the subject, time-series data indicating data pointsshowing a pattern or trend of attributes or capability levels over time,current and previous activities of the subject, attempts or actions bythe subject to perform tasks as well as corresponding outcomes, futureactivities of the subject (e.g., actions inferred, planned, and/orscheduled) which can include potential future training of the subject,future physical changes to the subject (e.g., a hardware upgrade for adevice, a maintenance action for a vehicle, a surgical intervention fora person, etc.). These techniques, and others discussed below, allow thesystem to make predictions high accuracy and high precision, with theversatility to predict many types of subjects and types of readiness.

The one or more models can be machine learning models, for example, aneural networks or classifiers. Other types of models that may be usedinclude support vector machines, regression models, reinforcementlearning models, clustering models, decision trees, random forestmodels, genetic algorithms, Bayesian models, and Gaussian mixturemodels. Different types of models can be used together as an ensemble orfor making different types of predictions. Other types of models can beused, even if they are not of the machine learning type. For example,statistical models and rule-based models can be used.

The techniques discussed herein are useful in many different industriesand technical fields. For example, computer systems may use thetechniques to predict completion of a complex task, such as compilingand linking a large software application. Rather than using milestonesor an equation to predict when the task will be complete, models can betrained to predict completion times using training data examplesindicating the steps needed for similar tasks (e.g., compiling othersoftware applications) and their actual completion times. This allowsthe system to model many other factors, such as the likelihood oferrors, severity of errors, the contributions of delays when user inputmay be needed in the process, and so on. Because the model(s) can takeinto account these and other factors, as observed over a range ofdifferent actual examples, predictions from the models regardingcomputing tasks can be much more accurate and precise than using typicaltechniques.

The techniques are also applicable to industrial settings. For example,the system may be used to predict when a factory under construction willbe achieve a level of capability, such as being fully functional toproduce a component or to reach production at a desired volume or rate.Models can be trained based on examples of other actual factoryconstruction projects, including not only the final time of completionbut the progressions of progress and the events during the constructionprocess (e.g., involving change requests, variability in completiontiming, differing amounts of resources available for different projects,etc.). As a result, the models can be used to generate accuratepredictions about the current and future state of a project, as well asshow how a project's progression relates to other prior projects, andwhat steps can be taken to improve readiness to achieve its purpose.

The system is also applicable to the medical field, where informationabout many different patients can be used to determine when a patient isready to begin or discontinue a treatment (e.g., physical therapy,surgery, medication, digital therapeutics, etc.). For example,information about many individuals health progression over time can becaptured, along with information about the patients and their lifestyle.Treatments and other activities can also be identified. This informationcan be used, for example, to model the progression of recovery from ahealth condition to predict when the patient will be ready todiscontinue or lessen treatment, or to model the progression of adisease (e.g., a degenerative or progressive disease) to predict whenthe patient will be ready for further intervention (e.g., surgery,higher dose of medication, a change in medication, etc.). Moregenerally, the techniques may be used to model and predict when anindividual will, for example, achieve capability to perform a physicaltask (e.g., a task being trained through occupational therapy orrehabilitation), reach a desired level of proficiency at an activity(e.g., be able to run a mile within a target time duration), a reach acertain state of health (e.g., to lower blood pressure to a targetlevel), change a behavior (e.g., to reduce or stop smoking), be readyfor a treatment (e.g., be ready to begin swallow therapy after throatcancer surgery), and so on. Beyond the medical field, the techniques canbe used for modeling and predictions of training individuals to acquireskills, abilities, health, wellness, etc. along any of variousdimensions (e.g., physiological, psychological, mental, intellectual,emotional, financial, spiritual, etc.).

The system is applicable to research efforts, as a tool to assistresearchers and facilitate scientific discovery. In this case, thesystem may be leveraged to benefit researchers in designing, monitoring,and enhancing a study such as a clinical trial, a cohort study, or otherresearch endeavor. For example, the subject whose readiness is monitoredmay be a research study that is ongoing or is under development. Forexample, the system may be used to predict the likelihood of completionof the study by individual participants (e.g., compliance with studyrequirements or adherence to a plan). As another example, the system maybe used to predict the likelihood that a study having certain parameterswill to reach a desired outcome or progress to a desired level (e.g.,have a minimum number of participants reporting a target level of datafor a target amount of time, or acquire an amount of data to provide astatistically significant result, etc.). The system may be used topredict whether a study specification and/or proposed cohort will likelyachieve study objectives. In other words, the readiness criteria for aresearch study can be, among other options, a readiness of a study toacquire a desired set of data or a readiness to be able to answer aresearch question. For example, the models can be used to predict whenparameters for a study being designed and/or composition of a cohortbeing defined are ready so that, if used, are predicted to achieve atleast a minimum level of confidence that the study requirements will besuccessfully completed by at least a minimum number of participants.

The system can be made available through various different methods. Forexample, the functionality to track and enhance readiness, as well as toobtain and implement interventions, may be initiated or managed bydifferent parties. One example is a consumer-driven approach, whereindividual users can seek access to the system to obtain support inimproving their health, athletic performance, skills, or othercapabilities. The data collection, predictions, and recommendations ofthe system can be provided through a web page, a web application, acomputer application, a mobile device application, etc. For example, auser may download an application for a mobile device, e.g., from anapplication store, and the application can provide the functionality forsetting goals of target capability, predictions for achieving thatcapability, and interventions over time to help the user reach thetarget. Another example for providing the features of the application isthrough an invitational model. For example, the system can be madeavailable in a business-to-business offering, so that employees can beinvited to participate as part of an employee wellness program. Asanother example, a doctor may invite or instruct a patient to use thesystem through a clinical prescription or recommendation.

There are many ways that the system can be used to provide advantages todifferent parties. For example, precision predictions can be useful fora subject (e.g., an individual or participant whose readiness ismonitored and predicted) and for an observer (e.g., a coach, employer,insurer, doctor, researcher, etc.). Different parties may use the systemin different ways to obtain different outcomes. An individual may desireto use the system to improve his or her health or skill. An employer maydesire to use the system to increase the wellness of employees, to buildteams with desired capabilities, or qualify for affordable insurance. Aresearcher may be interested in obtaining highly accurate predictivemodels, with interventions for individuals being of less importance.

For example, the system may be selected by an individual to producebenefits for the individual, e.g., improved predictability andassistance in reaching a better state of health, wellness, orcapability. When the focus is the individual, the key outcome of thesystem may be interventions that can assist the user to improve,including by personalizing and tailoring the individual's training plan,progression, or goals according to the values the individual holds. Forinstance, some users may select to increase human performance traits(e.g., increased strength, decreased weight, increase muscle to fatratios). Other users may select to obtain an improved life span andquality of life.

As another example, the system can be used by a coach, doctor, physicaltherapist, or other person as an aid to help others improve theircapabilities. The system's ability to predict capability levels, as wellas predict a personalized progression of capability over time, can beparticularly valuable. Similarly, the system can evaluate plans topredict the likely effects, enabling a coach or other person to simulatehow different plans are likely to affect the progression of differentindividuals. The system can also provide value by indicating specificrecommendations that are individually tailored to each individual whosecapability is being assessed.

As another example, the system can be used by a business or otherorganization to achieve benefits for the organization. For a sportsteam, the key outcome may be to adjust components across a group ofindividuals, to create the best team as a whole. For a business, theoutcome or readiness that is sought may be to qualify for an affordableinsurance plan when considering a representative group model, byachieving favorable averages across certain traits.

As another example, the system can be used in research as a tool used inorder to achieve scientific discovery gains. When the focus is theresearch, the outcome may be the creation of a highly accurate model andthe relationships that the model highlights, and not necessarily theinterventions that could be created for individuals. With a researchfocus, models may suggest changes to a particular study, such as adifference in cohort, demographics, or other aspects, that lead toexpanding the research study to obtain improved accuracy and precisionin the model. The improved models generated in this manner, or therelationships identified in this way, may in turn be used to betterprovide predictions to individuals, organizations, and others.

In general, the techniques in this document can compare measurementdata, for example, self-reported data or data captured by a sensor ordevice, and then select an algorithm or model in order to score anindividual or group for their effective performance readiness. When thesubject is a person, the model or algorithm can be used to score orpredict human performance readiness. The precision readiness scoringprovides the ability then to perform an intervention eitherautomatically or through manual review in order to bolster adherence andoutcomes and further improve readiness. The system optionally supports abring-your-own-algorithm (BYOA) option, in which third-party models andsystems can be connected or integrated to enhance or extend thecapabilities of the system.

In general, a computer system receives data indicating attributes and/oractivities of a subject and uses this data to determine a predictionwith respect to the subject. The prediction may be made with respect toone or more readiness criteria, which specify a target level ofreadiness that the subject is progressing toward. The prediction may beprovided in the form of a readiness score indicating of a subject'scurrent readiness with respect to the criteria or the subject's futurereadiness (e.g., at a future date, or after the passage of apredetermined time such as one month in the future). Estimates ofcurrent capability of a subject are still considered to be a prediction,for example, because the current level of capability is often notdirectly measurable or verifiable based the input data, and so the levelis inferred based on contextual and historical factors, for example, bytrained machine learning models. The system may predict, for example,(i) whether the subject is likely to reach a threshold level ofreadiness or other outcome, (ii) whether the subject is likely to reachthe target outcome (e.g., if the subject is capable of reaching thetarget, potentially given some constraints or conditions), and/or (iii)a time when the subject will reach the threshold readiness score orother performance criteria.

In some implementations, the system may use the data about the subjectgenerate a trend or progression of capability of the subject towardreaching the one or more readiness criteria. The system may use thegenerated trend in determining a readiness score, a likelihood ofreaching a threshold readiness score or other performance criteria,whether the threshold readiness score or other performance criteria willbe reached by a certain time, whether the threshold readiness score orother performance criteria is likely to be reached at all, and/or a timeto reach the threshold readiness score or other performance criteria.For example, the system may compare the trend (e.g., curve orprogression of readiness) for a subject with the trends of othersubjects to determine how the subject compares. From this comparison,the system can determine how to change a plan for the user to reachreadiness, such as by adding, altering, or removing training activities,changing communication patterns, suggesting physical changes to asubject (e.g., altering device configuration, supporting behaviorchanges for diet and exercise for a person, etc.), etc.

In one general aspect, a method performed by one or more computersincludes: accessing, by the one or more computers, a database to obtainstatus data that indicates activities or attributes of a subject, thestatus data comprising data provided from by an electronic device to theone or more computers over a communication network; deriving, by the oneor more computers, a set of feature scores from the status data for thesubject, the set of feature scores including values indicative ofattributes or activities of the subject; providing, by the one or morecomputers, the set of feature scores to one or more models that havebeen configured to predict readiness of subjects to satisfy one or morereadiness criteria; generating, by the one or more computers and basedon processing performed using the one or more models and the set offeature scores, a prediction indicating (i) a predicted time that thesubject will achieve readiness to satisfy the one or more readinesscriteria or (ii) whether the subject will achieve readiness to satisfythe one or more readiness criteria; and based on the prediction,providing, by the one or more computers, output data that is configuredto alter a user interface of a device or to alter interaction of adevice with the subject.

In some implementations, the one or more models comprise one or moremachine learning models.

In some implementations, the readiness criteria comprise at least oneof: a physical state of the subject; a functional state of the subject;or a level of capability of the subject to perform a task.

In some implementations, the one or more models comprise at least one ofa neural network, a support vector machine, a classifier, a regressionmodel, a clustering model, a decision tree, a random forest model, agenetic algorithm, a Bayesian model, or a Gaussian mixture model.

In some implementations, the one or more models have been trained basedon training data indicating (i) activities or attributes of othersubjects and (ii) outcomes for the other subjects with respect to thereadiness criteria, and the training uses the respective progressions ofreadiness of the other subjects over time to configure the one or moremodels to predict readiness of subjects to satisfy one or more readinesscriteria.

In some implementations, the database comprises data generated for thesubject over a period of time, the status data comprising informationabout activities or attributes of the subject at multiple points intime. The set of feature scores derived from the status data comprisemeasures of the activities or attributes of the subject at each of themultiple points in time.

In some implementations, the subject is a device, a system, a model, aresearch study, a hardware component, a software module, anorganization, a team, or an individual.

In some implementations, the set of feature scores is based on sensordata that is acquired by one or more sensors during one or moreactivities of the subject or that indicates one or more attributes ofthe subject.

In some implementations, the subject is an individual, and whereinreceiving status data comprises receiving at least one of heart ratedata for the subject, oxygen saturation data for the subject, dataindicating an exercise distance for the subject, data indicating anexercise intensity for the subject, or data indicating a duration of anexercise for the subject.

In some implementations, generating the prediction comprises generatingoutput indicating a predicted time that the subject will achievereadiness to satisfy the one or more readiness criteria.

In some implementations, generating the prediction comprises generatingoutput indicating whether the subject will achieve readiness to satisfythe one or more readiness criteria.

In some implementations, generating the output indicating whether thesubject will achieve readiness to satisfy the one or more readinesscriteria comprises: generating output indicating whether the subjectwill achieve readiness to satisfy the one or more readiness criteria,the output being generated to predict an outcome based on one or moreconstraints that comprise a limit on at least one of: a time to achievereadiness; resources available to the subject; or a training plan forthe subject.

In some implementations, providing the output data comprises providingat least one of: a score indicative of the prediction for display on auser interface; an indicator of the prediction for display on a userinterface; visualization data, generated using the prediction, forillustrating a timeline or trend of predicted readiness of the subjectover time; a recommendation, determined based on the prediction, of anaction to improve or accelerate acquisition of readiness of the subjectto satisfy the one or more readiness criteria; one or more interactiveuser interface controls to alter one or more planned actions forassisting the subject to achieve readiness to satisfy the one or morereadiness criteria; one or more interactive user interface controls toselect from among multiple options for assisting the subject to achievereadiness to satisfy the one or more readiness criteria; or anindication of a predicted change in readiness of the subject, relativeto a level of readiness indicated by the prediction, for each of one ormore actions with respect to the subject.

In some implementations, the method includes: tracking, by the one ormore computers, (i) attributes or activities of each of multiplesubjects over a period of time and (ii) changes in levels of capabilityof the multiple subjects over the period of time; and training, by theone or more computers, the one or more models based on the tracked data.

In some implementations, the subject is a group of individual subjects,wherein the one or more readiness criteria comprises one or more groupreadiness criteria, and wherein receiving status data comprisesreceiving activity data indicating activities of individual subjects inthe group. The method further includes: generating, based on theactivity data, a readiness score for each of the individual subjects inthe group, the readiness scores indicating predictions of the respectivecapabilities of the individual subjects in the group; and generating agroup readiness measure that indicates a predicted ability of the groupof individual subjects to collectively satisfy the one or more groupreadiness criteria.

In some implementations, providing output comprises providing, based onthe group readiness measure, output indicating (i) a predicted time thatthe group will achieve readiness to satisfy the one or more groupreadiness criteria or (ii) a likelihood or prediction whether the groupwill achieve readiness to satisfy the one or more group readinesscriteria given one or more constraints.

In some implementations, the method includes: receiving new activitydata indicating activities of the subjects in the group; generating,based on the new activity data, a new readiness score for each of thesubjects in the group; generating a new group readiness measure; andproviding, based on the new group readiness measure, new outputindicating (i) a new predicted time that the group will achievereadiness to satisfy the one or more group readiness criteria or (ii) anew likelihood or prediction whether the group will achieve readiness tosatisfy the one or more group readiness criteria given one or moreconstraints.

In some implementations, the method includes determining that the newoutput indicates that (i) the new predicted time is less than thepredicted time, or (ii) that the new likelihood is higher than thelikelihood.

In some implementations, the method includes determining that the groupreadiness measure does not satisfy a threshold; and in response todetermining that the group readiness measure does not satisfy thethreshold, performing at least one of: removing from the group one ormore subjects determined to have individual readiness scores that areless than an average of individual readiness scores for subject in thegroup; or adding to the group one or more subjects having individualreadiness scores that are greater than an average of individualreadiness scores for subject in the group.

In another general aspect, a method performed by one or more computersincludes: accessing, by the one or more computers, a database to obtainstatus data that indicates activities or attributes of a subject, thestatus data comprising data provided by an electronic device to the oneor more computers over a communication network; generating, by the oneor more computers, one or more readiness scores indicating a level ofcapability of the subject to satisfy one or more readiness criteria, theone or more readiness scores being generated using one or more modelstrained based on data indicating patterns of acquiring readiness byother subjects; accessing, by the one or more computers, data indicatingmultiple candidate actions for improving capability of the subject tosatisfy one or more readiness criteria; selecting, by the one or morecomputers, a subset of the candidate actions for the subject based onthe one or more readiness scores generated using the one or more models;and providing, by the one or more computers, output configured to (i)cause one or more of the actions in the selected subset to be performedby the one or more computers or another system or (ii) cause anindication one or more of the actions in the selected subset to bepresented on a user interface of the one or more computers or anothersystem.

In some implementations, the one or more models comprise one or moremachine learning models.

In some implementations, the one or more readiness criteria comprise atleast one of: a physical state of the subject; a functional state of thesubject; or a level of capability of the subject to perform a task.

In some implementations, the one or more models comprise at least one ofa neural network, a support vector machine, a classifier, a regressionmodel, a clustering model, a decision tree, a random forest model, agenetic algorithm, a Bayesian model, or a Gaussian mixture model.

In some implementations, the one or more models have been trained basedon training data indicating (i) activities or attributes of othersubjects and (ii) outcomes for the other subjects with respect to thereadiness criteria. The training uses the respective progressions ofreadiness of the other subjects over time to configure the one or moremodels to predict readiness of subjects to satisfy one or more readinesscriteria.

In some implementations, the database includes data generated for thesubject over a period of time, the status data comprising informationabout activities or attributes of the subject at multiple points intime; and the one or more readiness scores are generated using the oneor more machine learning by providing, to the one or more models, a setof feature scores, derived from the status data, that includes measuresof the activities or attributes of the subject at each of the multiplepoints in time.

In some implementations, the subject is a device, a system, a model, aresearch study, a hardware component, a software module, anorganization, a team, or an individual.

In some implementations, the set of feature scores is based on sensordata that is acquired by one or more sensors during one or moreactivities of the subject or that indicates one or more attributes ofthe subject.

In some implementations, accessing the data indicating the multiplecandidate actions includes: accessing data indicating (i) a plurality oftraining options for the subject and (ii) indications of how therespective training options are predicted to affect the readiness of thesubject to satisfy the one or more readiness criteria.

In some implementations, accessing the data indicating the multiplecandidate actions includes accessing data indicating candidate actionscomprising at least one of: changing a frequency, number, or intensityof planned training activities for the subject; changing an allocationof resources for the subject; changing a type of training for thesubject; changing an assignment of an individual to assist the subjectin achieving readiness; or providing, for output by a client device, anotification indicating the level of capability of the subject tosatisfy one or more readiness criteria that is indicated by the one ormore readiness scores.

In some implementations, accessing the data indicating the multiplecandidate actions includes: generating one or more candidate actionsbased on information indicating available resources for training thesubject and at least one of: information about the subject from thedatabase; data indicating actions taken to improve readiness of thesubject; or data indicating one or more planned future actions toimprove readiness of the subject.

In some implementations, selecting the subset of the candidate actionsincludes: determining a score for each of the candidate actions, thescores being indicative of a predicted effect of the respectivecandidate actions on readiness of the subject to satisfy the one or morereadiness criteria; and selecting the subset based on the scores for thecandidate actions.

In some implementations, the scores for the candidate actions aredetermined based on data indicating changes in attributes orcapabilities of one or more other subjects after actions correspondingto the candidate actions are performed for the one or more othersubjects.

In some implementations, the scores for the candidate actions aredetermined based on data indicating previous activities of the subjectand changes in attributes or capabilities of the subject after theprevious activities.

In some implementations, the scores for the candidate actions aredetermined based on current attributes of the subject.

In some implementations, the scores for the candidate actions aredetermined based on planned or scheduled activities of the subject.

In some implementations, providing the output includes using theselected one or more actions in the selected subset to change a plan offuture activities to increase readiness of the subject to satisfy theone or more readiness criteria.

In some implementations, providing the output includes: providing arecommendation, for presentation on a user interface of a client device,of one or more of the actions in the selected subset.

In some implementations, the method includes providing, for presentationon the user interface of the client device, at least one of: one or moreinteractive user interface controls configured to initiate one or moreof the actions in the subset in response to user interaction with theone or more interactive user interface controls; one or more scoresindicative of a predicted change to the readiness of the subject tosatisfy the one or more readiness criteria; or visualization data toillustrate a timeline or trend indicating a progression of readiness ofthe subject to satisfy the one or more readiness criteria if one or moreof the selected actions are performed.

In another general aspect, a method performed by one or more computersincludes: accessing, by the one or more computers, data designating agroup comprising multiple individual subjects; receiving, by the one ormore computers, status data indicating attributes or activities of eachof the subjects in the group, where at least some of the status dataincludes information provided to the one or more computers over acommunication network by the subjects or from electronic devicesassociated with the subjects; generating, by the one or more computers,a group readiness measure based on the status data using one or moremodels, where the group readiness measure indicates a predicted abilityof the group to satisfy one or more group readiness criteria; and basedon the group readiness measure, providing, by the one or more computers,output data that is configured to alter a user interface of a device orto alter interaction of a device with one or more of the subjects in thegroup.

In some implementations, generating the group readiness measureincludes: generating an individual readiness score for each of themultiple subjects. The individual readiness scores indicate predictionsof the respective capabilities of the subjects in the group; andgenerating the group readiness measure based on the individual readinessscores for the subjects in the group.

In some implementations, the individual readiness scores respectivelyindicate predicted readiness of individual subjects to satisfy one ormore individual readiness criteria, and the individual readinesscriteria are different from the group readiness criteria.

In some implementations, generating the individual readiness score foreach of the multiple subjects includes: for each of the multiplesubjects: providing, to one or more models, a set of feature scores forthe subject derived from status data for the subject obtained from adatabase of attributes or activities of subjects; and receiving anoutput that is generated for the subject in response to the one or moremodels receiving the set of feature scores for the subject, where theindividual readiness score for the subject is based on the output.

In some implementations, generating the group readiness measure based onthe individual readiness scores for the subjects in the group includes:providing the individual readiness scores to one or more models trainedto generate a group readiness measure based on examples of groupreadiness measures and corresponding individual readiness scores formembers of the group; and determining the group readiness measure basedon the processing of the one or more models performed in response toreceiving the individual readiness scores.

In some implementations, the group readiness measure is an average ofindividual readiness scores for each of the subjects in the group.

In some implementations, the group readiness measure indicates apredicted current readiness of the group to satisfy the one or moregroup readiness criteria.

In some implementations, the group readiness measure indicates apredicted readiness of the group to satisfy the one or more groupreadiness criteria at a future time.

In some implementations, the method includes adding or removing one ormore subjects from the group based on the prediction or based onindividual readiness scores for the subjects in the group.

In some implementations, the method includes reassigning a particularsubject from the group to a different group based on the prediction orbased on an individual readiness score for the particular subject.

In some implementations, the group readiness criteria correspond to atleast one of: a physical state of the subjects; a functional state ofthe subjects; or a level of capability of the subjects to perform atask.

In some implementations, the one or more models comprise at least one ofa neural network, a support vector machine, a classifier, a regressionmodel, a clustering model, a decision tree, a random forest model, agenetic algorithm, a Bayesian model, or a Gaussian mixture model.

In some implementations, the one or more models have been trained basedon training data indicating (i) activities or attributes of othersubjects and (ii) outcomes for the other subjects with respect tocorresponding readiness criteria. The training can use the respectiveprogressions of readiness of the other subjects over time to configurethe one or more models to predict readiness of subjects to satisfy oneor more readiness criteria.

In some implementations, the subjects comprise a device, a system, anetwork, a model, a research study, a hardware component, a softwaremodule, an organization, a team, or an individual; and the status datais based on sensor data that is acquired by one or more sensors duringone or more activities of the subjects or that indicates one or moreattributes of the subjects.

In some implementations, generating the group readiness measure includesgenerating output indicating a predicted time that the group willachieve readiness to satisfy the one or more group readiness criteria.

In some implementations, generating the group readiness measure includesgenerating output indicating whether the group will achieve readiness tosatisfy the one or more group readiness criteria.

In some implementations, generating the output indicating whether thegroup will achieve readiness to satisfy the one or more readinesscriteria includes: generating output indicating whether the group willachieve readiness to satisfy the one or more group readiness criteria,the output being generated to predict an outcome based on one or moreconstraints that comprise a limit on at least one of: a time to achievereadiness; resources available; or a training plan for the group or oneor more subjects in the group.

In some implementations, providing the output data includes providing atleast one of: a score indicative of the prediction for display on a userinterface; an indicator of the prediction for display on a userinterface; visualization data, generated using the prediction, forillustrating a timeline or trend of predicted readiness of the group orone or more subjects in the group over time; a recommendation,determined based on the prediction, of an action to improve oraccelerate acquisition of readiness of the group to satisfy the one ormore group readiness criteria; a recommendation, determined based on theprediction, of an action to improve or accelerate acquisition ofreadiness of one or more subjects in the group to satisfy one or moreindividual readiness criteria; one or more interactive user interfacecontrols to alter one or more planned actions for assisting the group toachieve readiness to satisfy the one or more group readiness criteria;one or more interactive user interface controls to select from amongmultiple options for assisting the group to achieve readiness to satisfythe one or more readiness criteria; or an indication of a predictedchange in readiness of the group, relative to a level of readinessindicated by the prediction, for each of one or more actions withrespect to the subject.

In another general aspect, a method performed by one or more computersincludes: receiving, by the one or more computers, status dataindicating attributes or activities of a subject, the status datacomprising sensor data acquired by one or more devices associated withthe subject; accessing, by the one or more computers, data indicatingmultiple different analysis techniques to assess readiness with respectto one or more readiness criteria; selecting, by the one or morecomputers, one of the multiple different analysis techniques based ondata received by the one or more computers that indicates a user inputto a user interface; using, by the one or more computers, feature dataderived from the status data to generate, according to the selectedanalysis technique, a measure of predicted readiness of the subject tosatisfy one or more readiness criteria; and providing, by the one ormore computers, output data that updates the user interface based on themeasure of predicted readiness of the subject or causes an interactionwith one or more devices based on the measure of predicted readiness ofthe subject.

In some implementations, the multiple different analysis techniquescomprise multiple different models.

In some implementations, the multiple different models comprise modelstrained based on different sets of training data or using differenttraining parameters.

In some implementations, the multiple different models comprise modelsof different types, and at least two of the different types are selectedfrom the group consisting of a neural network, a support vector machine,a classifier, a regression model, a reinforcement learning model, aclustering model, a decision tree, a random forest model, a geneticalgorithm, a Bayesian model, and a Gaussian mixture model.

In some implementations, at least some of the multiple models have beentrained to make predictions of readiness with different levels ofvariance.

In some implementations, at least some of the multiple models have beentrained to make predictions based on differing combinations of inputfeatures representing different sets of information about the subject.

In some implementations, the method includes: based on the accessed dataindicating the multiple different analysis techniques, providing, fordisplay on a user interface, an indication of each of the multipledifferent analysis techniques; and receiving data indicating userinteraction with the user interface that selects one of the multipledifferent analysis techniques. Selecting one of the multiple differentanalysis techniques includes selecting the analysis technique indicatedby the user interaction with the user interface.

In some implementations, the method includes comprising providing, fordisplay on a user interface, data indicating different predictions ofreadiness for a subject. The different predictions can each be generatedusing a different one of the multiple different analysis techniques.

In some implementations, the measure of predicted readiness of thesubject is generated based on data indicating planned or scheduledactivities to increase a capability of the subject.

In some implementations, the measure of predicted readiness of thesubject is generated based on data indicating previous activitiesperformed by the subject that were assigned to the subject to increase acapability of the subject.

In some implementations, each of the multiple analysis techniques usesexample data indicating (i) activities or attributes of other subjectsand (ii) outcomes for the other subjects with respect to at least one ofthe one or more readiness criteria. The analysis techniques can predictreadiness of subjects to satisfy one or more readiness criteria based onprogressions of readiness of the other subjects over time as indicatedin the example data.

In some implementations, providing the output data includes providing atleast one of: a score indicative of the measure of predicted readinessfor display on a user interface; an indicator of the measure ofpredicted readiness for display on a user interface; visualization data,generated using the measure of predicted readiness, for illustrating atimeline or trend of predicted readiness of the subject over time; arecommendation, determined based on the measure of predicted readiness,of an action to improve or accelerate acquisition of readiness of thesubject to satisfy the one or more readiness criteria; one or moreinteractive user interface controls to alter one or more planned actionsfor assisting the subject to achieve readiness to satisfy the one ormore readiness criteria; one or more interactive user interface controlsto select from among multiple options for assisting the subject toachieve readiness to satisfy the one or more readiness criteria; or anindication of a predicted change in readiness of the subject, relativeto a level of readiness indicated by the measure of predicted readiness,for each of one or more actions with respect to the subject.

Other embodiments of these and other aspects disclosed herein includecorresponding systems, apparatus, and computer programs encoded oncomputer storage devices, configured to perform the actions of themethods. A system of one or more computers can be so configured byvirtue of software, firmware, hardware, or a combination of theminstalled on the system that, in operation, cause the system to performthe actions. One or more computer programs can be so configured byvirtue having instructions that, when executed by data processingapparatus, cause the apparatus to perform the actions.

The details of one or more embodiments of the invention are set forth inthe accompanying drawings and the description below. Other features andadvantages of the invention will become apparent from the description,the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C are diagrams that illustrate an example system forgenerating precision predictions of readiness.

FIGS. 2A-2B are diagrams that illustrate example processes forpredicting readiness.

FIG. 3 is a diagram that illustrates an example process for assessing asubject.

FIGS. 4 and 5 are diagrams that illustrates examples of tables of datathat can be used for predicting or assessing readiness.

FIG. 6 is a diagram that illustrates an example system for generatingprecision predictions of readiness.

FIG. 7 is a diagram that illustrates the data used by an example systemfor generating precision predictions of readiness.

FIG. 8 is a diagram that illustrates an example scoring of subjects.

FIGS. 9A-9E are diagrams that illustrate example interfaces.

FIGS. 10-11 is a diagram that illustrates example interfaces.

FIG. 12 is a flowchart that illustrates an example process forpredicting readiness.

FIG. 13 is a flowchart that illustrates an example process for tailoringtraining to improve readiness.

FIG. 14 is a flowchart that illustrates an example process forpredicting readiness.

FIG. 15 is a flowchart that illustrates an example process forpredicting readiness of a group or a team.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

In some implementations, a system provides the ability to predict andimprove the readiness of a subject for a task or activity. In general,readiness refers to a state in which a subject has an ability to performan action or function. Readiness can thus refer to a capability of asubject, e.g., the power or ability to perform an action, operation,function, task, etc. Readiness may additionally or alternatively referto the physical and/or functional state of a subject that permits thesubject to perform in a certain way or at a certain level. Readiness canbe assessed with respect to one or more readiness criteria, e.g., apredetermined specification of capabilities that can be used as areference. As used herein, the use of the term “readiness” can refer toa state of satisfying applicable readiness criteria, such as by asubject possessing the capability to prospectively perform apredetermined task or activity.

Many types of subjects have varying states and varying capabilities overtime. For example, computers experience hardware configuration changesand software updates that may increase or decrease capability to performcertain types of operations. As another example, a building underconstruction undergoes many actions over time in order to reach fullcapability for its purpose, whether for housing, office space,industrial use, and so on. As another example, individual people vary intheir capabilities over time, especially as they are involved inprocesses intended to improve their capabilities, for example, physicaltraining, medical treatment, acquisition of a skill, etc. As anotherexample, organizations and teams often involve combinations of peopleand equipment, where the group of people and equipment available changesand the capabilities of each person or device may also vary over time,resulting in different levels of capability of the organization or teamas a whole.

To provide precision predictions of readiness of a subject, a computersystem can collect data about various subjects and how their readiness(e.g., capabilities) vary over time. This data can include attributes ofsubjects and activities of the subjects over time, as well as otherinformation indicating contextual information and other circumstances.With the data, the computer system can generate models, e.g., machinelearning models or rule-based models, that model the progression ofreadiness of subjects (e.g., the acquisition of capabilities by thesubjects) over time and the factors that enhance and detract fromreadiness. With these models, the computer system can use status dataabout a subject, e.g., attributes and/or activities of the subject, tomake predictions about the current and future readiness (e.g.,capabilities) of the subject. These predictions cam be made with respectto one or more readiness criteria, which specify a target level ofcapability for the subject to achieve. For example, the predictions canindicate a current level of readiness of the subject, a future level ofreadiness of the subject, a time (e.g., a date) that the subject isexpected to achieve readiness, an estimated trajectory of readinessimprovement by the subject, and whether the subject is likely to eversatisfy the readiness criteria. Other predictions can include how long asubject will maintain readiness after achieving readiness, a likelihoodthat readiness will be maintained (e.g., for a certain duration of timeor to a certain time), whether certain actions will improve themaintenance of readiness, and so on.

Predictions may be made based on certain constraints or assumptions usedfor purpose of the analysis, such as the subject following a currenttraining plan or having availability of certain limited resources (e.g.,time, equipment, power, materials, etc.). The computer system can usereadiness predictions and stored data to identify and recommend changesto enhance the readiness of a subject. For example, if the computersystem identifies that a subject is not predicted to achieve applicablereadiness criteria by a target time, the computer system may identifyinterventions to increase readiness. These interventions can be, atleast in some cases, activities, resources, communications, planchanges, and other actions that are personalized for the subject basedon the analysis of or models learned from data about the progression ofother subjects in acquiring readiness.

As an example, if a particular subject is predicted to lack readiness,the computer system can identify resources that were used by othersubjects that (i) achieved the same or similar readiness criteriaapplicable for the particular subject and (ii) had similar attributesand/or past activities as the particular subject. The computer systemcan then select and initiate actions that the computer system predictswill improve the readiness of the particular subject. These actions mayinclude changing resources available to the subject, recommending anaction to the subject or another person, initiating communications withdevices, causing changes to user interfaces, and so on. The computersystem can thus leverage a machine learning techniques to providerecommendations or interventions that are calculated to, for example,reduce the predicted time for the subject to meet the readiness criteriaand/or to increase the likelihood of the subject meeting the readinesscriteria.

FIGS. 1A-1C are diagrams that illustrate an example system 100 forgenerating precision predictions of readiness. FIG. 1 shows a computersystem 110 that includes functionality to predict readiness and provideinformation and adjustments to improve readiness. FIG. 1A shows variousfunctions of the computer system 110 and interactions to predict andenhance a diverse set of subjects 103 a-103 c. FIG. 1B shows anadditional example, in which the computer system collects data for andgenerates predictions for subjects 102 a-102 c that are people beingtrained. FIG. 1C shows functionality of the computer system 110 togenerate various types of predictions using different models, as well asto identify candidate actions and select actions to improve readiness ofsubjects.

As illustrated in FIG. 1, the computer system 110 has access to adatabase 125 and also communicates with a client device 105 over anetwork 160. The computer system 110 provides data to the client device105 to enable a user to set readiness criteria, manage subjects, alterreadiness plans (e.g., training plans, maintenance plans, etc.), viewpredictions, and otherwise interface with the computer system 110.

The system can be used to predict and enhance readiness of many types ofsubjects. Examples of subjects include, a device, a system, a network, afleet, a model (e.g., such as a machine learning model being trained), aresearch study (e.g., a research study being designed or is ongoing), ahardware component, a software module, an organization, a team, or aperson. In some implementations, the subject can be a computing device,a machine learning model, a robot, or other device that can expand itscapabilities (e.g., through hardware configuration changes, softwareupdates and configuration changes, machine learning training, etc.). Insome implementations, the subject is a person, a group of people, a teamor group of people, and so on who can improve their capabilities (e.g.,health, capabilities, skills, etc.) through various activities. In someimplementations, the subject may be a combination of various individualsubjects, such as a group of people along with various items ofequipment that collectively form a team or unit.

In the example of FIG. 1A, the subject 103 a is a computer system, thesubject 103 b is a person that uses a mobile device, and the subject 103c is a vehicle. Each of these different subjects can have a differentset of capabilities to be acquired or developed. For example, for thecomputer system 103 a, the readiness criteria may specify a target levelof capacity and throughput for data processing. For the person 103 b,the readiness criteria may specify achieving certain health indicators(e.g., weight, blood pressure, etc.) and/or abilities (e.g., ability torun a mile in less than a maximum time limit). For the vehicle 103 c,the readiness criteria may specify achieving a target emissions level orreaching full function over the course of repair work. The computersystem 110 can collect data about each of the subjects 103 a-103 c overtime and provide predictions and assistance for the subjects to reachtheir respective desired states or functional capabilities.

Some or all of the functions of the computer system 110 can be performedby a server system. In some implementations, functions of the computersystem 110 may be implemented on a cloud computing platform (e.g.,Amazon Web Services (AWS), Microsoft Azure, and so on).

The network 160 can include public and/or private networks and caninclude the Internet. The network 160 may include wired networks,wireless networks, cellular networks, local area networks, wide areanetworks, etc.

The database 125 provides data storage and retrieval capabilities to thecomputer system 110. The database 125 may include, but is not requiredto include, one or more of a relational database, a centralizeddatabase, a distributed database, a data warehouse, a noSQL database, anobject-oriented database, a graph database, a cloud-computing database,a data repository, a data lake, etc.

The computer system 110 is configured to receive data about subjectsfrom a variety of sources and store the data in the database 125. Thecomputer system 110 can monitor subjects and their progress in achievingreadiness and collect the data in the database 125. The collected datamay come from the subjects themselves, or from devices associated withthe subjects. The collected data may come from a variety of sensors,e.g., temperature sensors, cameras, microphones, accelerometers,pressure sensors, optical sensors, and so on. Some data about thesubject may be provided manually by a user, for example, through a userinterface, a text message, an email message, a voice input, etc.

Some data collected in the database 125 is obtained as devicesautomatically send reports of conditions detected, for example, inresponse to detecting certain conditions or periodically (e.g., withconsistent updates at a defined interval). Some data collected in thedatabase 125 can be obtained in response to interactions initiated bythe computer system 110, for example, to request sensor data, to providea form or survey to a user, to initiate a test or evaluation of asubject, and so on. As the computer system 110 obtains data aboutsubjects and learns the factors that are predictive of readiness, thecomputer system 110 may adjust the types of data that are collected fordifferent subjects. For example, the computer system 110 may identifycertain types of data or certain survey questions as relevant to aparticular capability based on the collected data for some subjects, andin response may instruct devices to collect those types of data forother subjects for which the capability is also monitored.

The system 100 allows for the accurate and efficient evaluation ofsubjects. For example, by collecting sensor data from a variety ofsensing devices, the system 100 can obtain wide range of data that moreaccurately represents the performance of the subject. Accuracy is alsoincreased by collecting data repeatedly over a period of time, whichallows a more accurate view of the progression of readiness than singleobservations. This can allow for determining a current performance of asubject with higher accuracy, and the more accurate tracking of thesubject's performance over time, which can be used in developinghistorical trends.

Using the data in the database 125, the computer system can generatemodels 109 that can be used to make predictions about the current andfuture readiness of subjects. For example, a model generation module 113can generate and train machine learning models, statistical models,rule-based models, and other types of models. Examples include neuralnetworks, support vector machines, classifiers, reinforcement learningmodels, regression models, clustering models, decision trees, randomforest models, genetic algorithms, Bayesian models, and Gaussian mixturemodels. The models 109 can be generated for the readiness criteria thata user provides, allowing the types of predictions and criteria forassessing readiness to be customized for each application.

The readiness criteria can be predetermined standards that indicate atarget state or level of capability to be achieved. For example, thereadiness criteria may specify a physical or functional state that asubject should achieve. As another example, the readiness criteria mayspecify a task or activity that the subject should be able to perform.Thus, the readiness criteria can serve as a reference level or thresholdlevel of capability against which the abilities of the subject aremeasured.

Once the models 109 have been generated, the computer system 110 usesthe models 109 to generate predictions 123 of readiness for subjects.For example, the computer system 110 includes a scoring and predictionmodule 114 that can generate various types of predictions. One example,is a predicted completion readiness time (CRT), which can represent apredicted time that the subject will reach readiness with respect topredefined readiness criteria. Another example is a future readinessprobability (FRP), which can represent a likelihood or probability thatthe subject will reach readiness at a time in the future (e.g., aspecific date or time in the future, or after a particular amount oftime has passed), potentially according to some constraints orassumptions factored in. Another example is an altered course successrate (ACS), which can indicate a likelihood that a change in thetraining or development of the subject will succeed in allowing thesubject to reach a desired level of readiness by a certain time.

Beyond predicting readiness, the system 100 can improve the likelihoodof subjects reaching particular goals and/or the time it takes forsubjects to reach those goals. Based on sensor data collected for asubject and the leveraging of historical trends and/or machine learningmodels, the system 100 can determine the likelihood of the subjectreaching their goal(s) and/or the time it will take to reach theirgoal(s). The system 100 can further identify one or more recommendationsto improve the likelihood of success and/or to reduce the time to reachthe goal(s), and may provide one or more of these recommendations to thesubject and/or may automatically implement one or more of theserecommendations into a program for the subject.

As an example, a plan analysis module 115 can be used to evaluate atraining or development plan for a subject and select interventions thatare predicted to improve readiness of the subject. For example, thedatabase 125 may indicate available resources, training options, andother actions that can improve readiness. The computer system 110 canuse data characterizing the available candidate options, as well as theinformation about the subject in the database, to select interventionsthat are customized for the subject and the subject's unique progressionin acquiring readiness to satisfy the applicable readiness criteria.

The system 100 can further improve the likelihood of subjects reachingtheir goals and/or the time it takes for subjects to reach their goalsby evaluating multiple subjects and grouping the subjects based on theirevaluations. When grouping subjects, the system 100 may group subjectsinto teams in order to maximize the number of teams that meet a groupcriteria, ensure that all teams meet a group criteria, and/or togenerate teams that are likely to perform similarly (e.g., therebyavoiding the creation of teams that are likely to underperform). Thesystem 100 may do this by organizing subjects into teams such that theaverage readiness score each of the teams is the same or similar, thatthe average readiness score of each of the teams is meets a thresholdamount as indicated by group criteria, and/or that the majority of theteams have an average readiness score that meets a threshold amount asindicated by group criteria. However, the system 100 may organizesubjects in more thoughtful ways, such as looking at specific weaknessesand strengths of subject subjects and coupling those subjects weak in aparticular area with subjects who are strong in that same area.

Referring to FIG. 1B, the system 100 includes various sensing devices104 a-104 c that collect sensor data 106 a-106 c from multiple subjectsor subjects 102 a-102 c, and a computer system 110. One or more of thesensing devices 104 a-104 c may communicate with the computer system 110over a network 160. One or more of the sensing devices 104 a-104 c maycommunicate with the computer system 110 through an intermediary device,e.g., a smart phone, a laptop computer, a desktop computer, or the like.

The sensing devices 104 a-104 c can include can include computing and/orwearable devices having one or more sensors. The sensing devices 104a-104 c can include equipment having one or more sensors, such asworkout equipment. The sensing devices 104 a-104 c may include, forexample, a smart phone (e.g., having one or more of a GPS unit, a heartrate sensor, a clock, etc.), a smart watch or band (e.g., having one ormore of a GPS unit, an optical heart/PPG rate sensor, an electricalheart rate/EKG sensor, a temperature sensor, a biochemical sensor, aclock, etc.), a chest strap (e.g., having a heart rate sensor), an earsensor (e.g., having an oxygen sensor), a wrist sensor (e.g., having oneor more of an oxygen sensor, a heart rate sensor, etc.), a finger sensor(e.g., having one or more of an oxygen sensor, a heart rate sensor,etc.), a treadmill (e.g., having a heart rate sensor, one or moresensors to track the minimum, maximum, and/or average speed during asession, one or more sensors to track the minimum, maximum, and/oraverage incline during a session, a clock, etc.), an elliptical or bike(e.g., having a heart rate sensor, one or more sensors to track theminimum, maximum, and/or average RPM, one or more sensors to track theminimum, maximum, and/or average resistance, a clock, etc.), etc.

In the example of FIG. 1B, the system 100 is evaluating three subjects102 a-102 c. The subjects 102 a-102 c may be, for example, persons suchas consumers, recruits, employees, potential employees, patients,trainees, etc.

The first subject 102 a is wearing the sensing device 104 a that iscollecting sensor data 106 a from the subject 102 a. Here, the sensingdevice 104 a is a smart band and collects data such as, for example, thesubject 102 a's current, minimum, maximum, and/or average heart rate(e.g., resting heart rate, heart rate while exercising, etc.); theminimum, maximum, and/or average time and distance that the subject 102a runs for in a given exercise session; the frequency that the subject102 a runs/exercises; etc.

The sensing device 104 a may send the sensor data 106 a that it collectsto the computer system 110 over the network 160. The sensor data 106 amay represent the sensor data that the sensing device 104 a collectsover a particular time period (e.g., an hour, six hours, twelve hours, aday, a week, etc.). The sensor data 106 a may be sent to the computersystem 110 as a single packet of information. Alternatively, portions ofthe sensor data 106 a may be sent to the computer system 110 as they areacquired. For example, each heart rate reading taken by the sensingdevice 104 a may be sent to the computer system 110 as soon as a resultis acquired. Accordingly, a first portion of the sensor data 106 a maybe sent to the computer system 110 prior to a different portion of thesensor data 106 a.

The second subject 102 b is wearing the sensing device 104 b that iscollecting sensor data 106 b from the subject 102 b. Here, the sensingdevice 104 a is a finger sensor and collects data such as, for example,the subject 102 b's current, minimum, maximum, and/or average heart rate(e.g., resting heart rate, heart rate while exercising, etc.); thesubject 102 b's current, minimum, maximum, and/or average oxygensaturation (e.g., through an oxygen sensor on the sensing device 104 bsuch as an oximeter); etc.

The sensing device 104 b may send the sensor data 106 b that it collectsto the computer system 110 over the network 160. The sensor data 106 bmay represent the sensor data that the sensing device 104 b collectsover a particular time period (e.g., an hour, six hours, twelve hours, aday, a week, etc.). The sensor data 106 b may be sent to the computersystem 110 as a single packet of information. Alternatively, portions ofthe sensor data 106 b may be sent to the computer system 110 as they areacquired. Accordingly, a first portion of the sensor data 106 b may besent to the computer system 110 prior to a different portion of thesensor data 106 b.

The third subject 102 c is using the sensing device 104 c that iscollecting sensor data 106 c from the subject 102 c. Here, the sensingdevice 104 c is a piece of exercise equipment, e.g., a treadmill. Thesensing device 104 c collects data such as, for example, the subject 102c's current, minimum, maximum, and/or average heart rate (e.g., heartrate while exercising); the minimum, maximum, and/or average time,distance, speed, and/or incline that the subject 102 a runs for in agiven exercise session; the frequency that the subject 102 a runs and/orwalks; etc.

The sensing device 104 c may send the sensor data 106 c that it collectsto the computer system 110 over the network 160. The sensor data 106 cmay represent the sensor data that the sensing device 104 c collectsover a particular time period (e.g., an hour, six hours, twelve hours, aday, a week, etc.). The sensor data 106 c may be sent to the computersystem 110 as a single packet of information. Alternatively, portions ofthe sensor data 106 c may be sent to the computer system 110 as they areacquired. Accordingly, a first portion of the sensor data 106 c may besent to the computer system 110 prior to a different portion of thesensor data 106 c.

In some implementations, data other than the sensor data 106 a-106 c issent to the computer system 110. This data may include self-reportedinformation 150 provided by the one or more of the subjects 102 a-102 c,third-party input 152, electronic health records 154, and genomic data156.

The self-reported information 150 may include mental performance data.For example, this data may include results of a test administered to thesubjects 102 a-102 c through one or more devices (e.g., smart phones,laptop computers, desktop computers, etc.). The self-reportedinformation 150 may include an indication of completed challenges ortasks, or an indication of a failure to complete challenges or tasks forchallenges or tasks that have been assigned to the subjects 102 a-102 c.The self-reported information 150 may include an indication of painlevels and intensities felt by the subjects 102 a-102 c, e.g., afterexercising, after taking medication, after completing a challenge, orthe like. The self-reported information 150 may be sent to the computersystem 110 over the network 160.

The third-party input 152 may include data submitted by a coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, supervisor of one or more of the subjects 102 a-102 c (e.g., acoach assigned to one or more of the subjects 102 a-102 c). Thethird-party input 152 may include indications of how the coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, or supervisor believes a corresponding subject is performing,e.g., they are top of their group or class, they are an over-performer,they are an under-performer, etc. As an example, the third-party input152 include feedback from a counselor on the mental fitness of one ormore of the subjects 102 a-102 c based on a most recent counselingsession. The third-party input 152 may be sent to the computer system110 over the network 160.

The electronic health records 154 may include health records that arestored on an EMR system or those that are stored on personal devices,e.g., smartphones, of the subjects 102 a-102 c. The electronic healthrecords 154 may be transferred to the computer system 110, e.g., overthe network 160, after the corresponding subject of the subjects 102a-102 c gives permission for the electronic health records 154 betransferred. In some implementations, the electronic health records 154include health information shown the table 500 of FIG. 5 such as, forexample, VO2Max. In some implementations, the electronic health records154 includes all or part of the genomics data 156.

The genomics data 156 may include data relating to the genes of thesubjects 102 a-102 c. The genomics data 156 may indicate whether any ofthe subjects 102 a-102 c suffer from any genetic diseases or otherissues, or whether they actually are likely to benefit from theirgenetic makeup. As an example, the genomics data 156 may include all orpart the table 400 of FIG. 4 such as, for example, the gene column.

When the computer system 110 receives the sensor data 106 a-106 c, thecomputer system 110 may provide the sensor data 106 a-106 c to a dataaggregation module 112. The data aggregation module 112 may organize thereceived sensor data 106 a-106 c. The data aggregation module 112 mayextract performance data from the received sensor data 106 a-106 c, andmay organize the extracted performance data. For example, the dataaggregation module 112 may organize the received sensor data 106 a-106 cand/or the extracted performance data by subject, by group of subjects(e.g., those that are applying for the same position, those that arebeing evaluated at the same time, those that have been assigned the samecoach or counselor, etc.), by time (e.g., may aggregate the data for asubject for a given week and separate it from data collected fromprevious weeks), by session, and/or by the type of performance data(e.g., mental vs. physical performance, heart rate data, run data,oxygen saturation data, etc.). For example, as shown, the dataaggregation module 112 has extracted performance data for each of thesubjects 102 a-102 c and has organized the performance data by subjectand by type of performance data, resulting in aggregated data 122.

The data aggregation module 112 may provide data (e.g., the aggregateddata 122, the sensor data 106 a-106 c, self-reported informationprovided by the subjects 102 a-102 c, etc.) to the scoring andprediction module 114. The scoring and prediction module 114 maydetermine one or more readiness scores for each of the subjects 102a-102 c based on received data. In determining a readiness score, thescoring and prediction module 114 may leverage one or more staticalgorithms. For example, the scoring and prediction module 114 may use aformula to calculate a readiness score for a person based on theirperformance or based on their performance with respect to their peers inone or more categories. In determining a readiness score, the scoringand prediction module 114 may leverage one or more machine learningmodels as will be discussed in more detail below with respect to FIG.1B. For example, the scoring and prediction module 114 may use a formulaand/or a first machine learning model for determining a currentperformance score for each of the subjects 102 a-102 c.

In calculating a readiness score for each of the subjects 102 a-102 c,the scoring and prediction module 114 may take into account the pastreadiness scores of the respective subject, the performance trends ofrespective subject, the frequency that the respective subject is meetingtheir respective challenges or goals, the amount of challenges or goalsthe respective subject has missed (e.g., may be used to determinewhether or not the subject is complying with their program),self-reported information 150 corresponding to the respective subject,third-party input 152 corresponding to the respective subject,electronic health records 154 corresponding to the respective subject,genomics data 156 corresponding to the respective subject, and/or thelike.

A readiness score may indicate a subject's mental performance orfitness, e.g., intelligence, mental fortitude, etc. For example, areadiness score may indicate a subject's ability to react, e.g., anability to push-through a particular challenge and stay on the currentpathway (e.g., which indicates mental fortitude of the subject), abilityto work around a particular challenge and continue the original trainingdesign (e.g., which indicates intelligence and performance of thesubject under pressure), ability to create an alternate an alternatepathway that results in the same performance testing outcome orobjective (e.g., which indicates intelligence and creativity of thesubject).

A readiness score may indicate a subject's physical performance orfitness. As an example, a readiness score may indicate a subject'sability to resist physical strain, e.g., the subject's physicalendurance.

In some implementations, the scoring and prediction module 114calculates a first readiness score that indicates that the physicalperformance of each of the subjects 102 a-102 c, and a second readinessscore that indicates the mental performance of each of the subjects 102a-102 c.

In some implementations, the scoring and prediction module 114calculates an overall readiness score for each of the subjects 102 a-102c. The overall score may be calculated using a physical performancescore and a mental performance score, e.g., by averaging the two scores.

In some implementations, in generating one or more scores for a givensubject, the scoring and prediction module 114 takes into account thereadiness criteria required of the subject and/or any other goalsassigned to the subject. The readiness criteria may require a particularlevel of mental performance, a particular level of physical performance,a particular overall level of performance, and/or a particularperformance in a particular performance category (e.g., must have aresting heart rate of no more than 70 BPM, must be able to run a 6:00min mile, must have scored greater than 85% in all math assessments,etc.). A goal may require the successful completion of a task orchallenge (e.g., running ten miles every week for a month, becoming atop 20% mental performer, etc.). The readiness criteria and/or goalsassigned to the subject may be provided by a coach, counselor,caregiver, care provider, specialist, employer, or potential employer.

In some implementations, the readiness criteria required of a subject isautomatically assigned by the computer system 110, e.g., based on thetype of subject (e.g., consumers, recruits, employees, potentialemployees, patients, trainees, etc.), based on the position that thesubject is applying for if a potential employee, based on the positionof the subject if an employee, etc.

In some implementations, one or more goals are automatically assigned tothe subject by the computer system 110, e.g., based on their past orcurrent readiness scores, based on the type of subject, etc.

The scoring and prediction module 114 may also predict one or morefuture readiness scores for each of the subjects 102 a-102 c. Inpredicting one or more future readiness scores, the scoring andprediction module 114 may leverage one or more machine learning models.The scoring and prediction module 114 may also make multiple predictionsfor each of the subjects 102 a-102 c corresponding to multiple futuretimes. For example, for a given subject, the scoring and predictionmodule 114 may predict a first readiness score at N time in the future(e.g., one week, one month, two months, one year, etc.), a secondreadiness score at N+1 time in the future, a third readiness score atN+2 time in the future, etc.

In some implementations, the scoring and prediction module 114 usesmultiple machine learning models to predict readiness scores. Forexample, as shown in the output 124 of the scoring and prediction module114, the scoring and prediction module 114 determines a currentreadiness score and a number of predictions for each of the subjects 102a-102 c. For example, for subject 102 a, the scoring and predictionmodule 114 determines a readiness score of 3/10, that the they are notlikely to reach set performance criteria (threshold readiness score of7/10) by February 1^(st), and that they are not likely to ever reach theset performance criteria. For subject 102 b, the scoring and predictionmodule 114 determines a readiness score of 7/10, that the they arelikely to reach set performance criteria (threshold readiness score of7/10) by February 1st, and that it will take no time for them to do sobecause they have already reached the set performance criteria. Forsubject 102 c, the scoring and prediction module 114 determines areadiness score of 5/10, that the they are likely to reach setperformance criteria (threshold readiness score of 7/10) by February1st, and that it will take them about sixty days to do so based.

In some implementations, in calculating an overall readiness score, thescoring and predication module 114 takes into account predicted scoresas well as the current and/or past readiness scores. For example, if asubject receives a score of 5/10 and the models predict an eventualscore of 7/10, the scoring and prediction module 114 may determine anoverall readiness score for the subject of 6/10.

In some implementations, the scoring and prediction module 114calculates a performance trend for each of the subjects 102 a-102 c. Incalculating a performance trend for each of the subjects 102 a-102 c,the scoring and prediction module 114 may leverage one or more staticalgorithms and/or one or more machine learning models. The performancetrend for a subject may indicate whether the subject is likely to reacha readiness by a particular time, whether the subject is likely ever toreach readiness, an expected time when the subject is likely to reachreadiness, and/or whether an intervention, e.g., a change, to thesubject's program is recommended.

In some implementations, the scoring and prediction module 114calculates a risk score in addition to a readiness score for each of thesubjects 102 a-102 c. In calculating a risk score for each of thesubjects 102 a-102 c, the scoring and prediction module 114 may leverageone or more static algorithms and/or one or more machine learningmodels. In calculating risk scores for the subjects 102 a-102 c, thescoring and prediction module 114 may take into account the currentreadiness score of the respective subject, the past readiness scores ofthe respective subject, the performance trends of respective subject,the frequency that the respective subject is meeting their respectivechallenges or goals, the amount of challenges or goals the respectivesubject has missed (e.g., may be used to determine whether or not thesubject is complying with their program), self-reported information 150corresponding to the respective subject, third-party input 152corresponding to the respective subject, electronic health records 154corresponding to the respective subject, genomics data 156 correspondingto the to the respective subject, and/or the like.

In some implementations, the computer system 110 groups the subjects 102a-102 c based on the output 124 of the scoring and prediction module114. The computer system 110 may group subjects into teams based ontheir readiness scores, e.g., in an effort to ensure that all teams meeta group criteria, to maximize the number of teams that meet a groupcriteria, to create teams having the same or similar average readinessscores, and/or in an effort to group subjects who perform poorly in agiven area with one or more other subjects who exceed in that same area.

For example, the computer system 110 may use the output 124 to determinethat subject 102 a (“Subject 1”), the lowest performing of the subjects102 a-102 c, should be grouped with subject 102 b (“Subject 2”), thehighest performing of the subjects. The computer system 110 may groupthe subject 102 c (“Subject 3”) with another middle performing subject,e.g., having a readiness score of 5/10. The computer system 110 maygroup the subjects in this manner to ensure that both teams meet a groupcriteria. The group criteria may be that the teams must have an averagereadiness score of 5/10. Here, the first team and the second team wouldboth have an average readiness score of 5/10. Accordingly, by groupingthe subjects in this manner, the computer system 110 has ensured thatboth teams meet the group criteria.

FIG. 1C depicts a more detailed view of the computer system 110. Asshown, the data aggregation module 112 provides subject data 116 to thescoring and prediction module 114. The subject data 116 may include allor part the aggregation data 122 shown in FIG. 1B, the sensor data 106a-106 c shown in FIG. 1B, and/or other data (e.g., self-reportedinformation, information from coaches or counselors, etc.). For example,the subject data 116 may include the aggregation data 122 relating tothe subject 102 a shown in FIG. 1A and/or the sensor data 106 a. Thesubject data 116 may be provided as input to one or more machinelearning models. For example, as shown, the scoring and predictionmodule 114 provides the subject data 116 as input to a first machinelearning model 130 (“Model A”), a second machine learning model 132(“Model B”), and a third machine learning model 134 (“Model C”).

The machine learning models 130, 132, and 134 may be training usinghistorical data. The historical data may include data from previouslyassessed subjects, groups of subjects, and/or teams of subjects. Thehistorical data may include an indication of the performance results forsubjects, groups of subjects, and/or teams of subjects. For example, thehistorical data may indicate whether the subjects, the groups ofsubjects, and/or teams of subjects met certain performance criteria bythe end of their respective programs, such as particular readinessscores by the end of their respective programs.

Collectively, the machine learning models 130, 132, and 134 may producethe output 140 for a given subject. The output 140 contains a graphwhere past, current, and predicted readiness scores have been plotted.

The output 140 also depicts readiness criteria that have been assignedfor the subject. The readiness criteria is a minimum acceptableperformance (“Minimum Needs”) and corresponds with a readiness score ofapproximately 4.5/10. The output 140 also provides an indication of howthe subject compares with top performing subjects (e.g., those subjectswith the top 20%, 15%, 10%, or 5% readiness scores) as indicated by the“Top Percentile” line that corresponds with a readiness score ofapproximately 7.5/10. Reaching this top level of performance may be goalassigned to the subject.

The output 140 further indicates a calculated readiness score 142 and acalculated risk score 144. As shown, the readiness score 142 of 7/10 mayactually be different than the current performance score 6/10. Forexample, the scoring and prediction module 114 may take into account thecurrent performance as well as the output of the machine learning models130, 132, and 134 in calculating the readiness score 142. Similarly, thescoring and prediction module 114 may take into account the currentperformance as well as the performance trends for the subject incalculating the readiness score 142. For example, the prior earnedperformance scores of 3/10, 4/10, and 5/10 indicate that the subject hasa positive performance trend. Accordingly, the subject's readiness score142 of 7/10 is greater than their performance score today of 6/10.

The machine learning models 130, 132, and 134 may output data indicatingthe readiness score 142 and/or the risk score 144. The readiness score142 may be calculated by the machine learning models 130, 132, and/or134, or by the scoring and prediction module 114 shown in FIGS. 1A-1Cafter receiving the output from the machine learning models 130, 132,and/or 134. In calculating the readiness score 142, or providing outputindicating the readiness score 142, the machine learning models 130,132, and/or 134 may take into account the subject's currentphysiological performance, psychological performance, or combination ofthe subject's physiological performance and psychological performance.The machine learning models 130, 132, and/or 134 may use the subject'scurrent physiological performance and/or psychological performance togenerate a base score. With respect to FIG. 1B, the physiologicalperformance and/or psychological performance may be determined from theobtained sensor data 106, the self-reported information 150, thethird-party input 152 (e.g., information provided by a coach, caregiver,potential employer, employer, supervisor, or the like), and/or theelectronic health records 154.

In calculating the readiness score 142, or providing output indicatingthe readiness score 142, the machine learning models 130, 132, and/or134 may also take into account the subject's past performances or pastperformance data. The subject's past performances may include one ormore past readiness scores, one or more past risk scores, previouslyobtained physiological performance data of the subject, previouslyobtained psychological performance of the subject, or the like. Themachine learning models 130, 132, and/or 134 may use the subject's pastperformances or past performance data to generate a performance trendfor the subject and then use the trend in calculating the readinessscore 142, or providing output indicating the readiness score 142. Forexample, if the machine learning models 130, 132, and/or 134 generate apositive performance trend (e.g., indicating that the subject'sperformance is improving), then the machine learning models 130, 132,and/or 134 may increase the base score in determining the readinessscore 142. If the machine learning models 130, 132, and/or 134 generatea negative performance trend (e.g., indicating that the subject'sperformance is worsening), then the machine learning models 130, 132,and/or 134 may decrease the base score in determining the readinessscore 142.

In some implementations, the machine learning models 130, 132, and/or134 weigh certain data or calculations differently in determining thereadiness score 142 (or the risk score 144). For example, with respectto FIG. 1B, the machine learning models 130, 132, and/or 134 may weighthe sensor data 106, the self-reported information 150, the third-partyinput 152, information within the electronic health records 154, and thesubject's performance trend(s) differently in calculating the readinessscore 142, or providing output indicating the readiness score 142.

In some implementations, in calculating the readiness score 142, orproviding output indicating the readiness score 142, the machinelearning models 130, 132, and/or 134 may take into account the genomicsdata 156 shown in FIG. 1A. As will be discussed in more detail belowwith respect to FIG. 4, the machine learning models 130, 132, and/or 134may improve or decrease a subject's readiness score based on theirpositive or negative effects arising from their genetic makeup. Themachine learning models 130, 132, and/or 134 may weigh the presence ofparticular genes differently, or may weigh the positive or negativeeffect of particular genes differently. For example, if the genomicsdata 156 indicates that the subject has a gene that makes them lesssensitive to pain, the machine learning model 130 may take into accountonly the positive effects (e.g., insensitivity to pain may providebenefits for the mental health of the subject, may result in them beingmore likely to complete their exercises, may make them more fit forcertain roles such as a soldier, etc.) of the gene and improve thesubject's readiness score, the machine learning model 132 may take intoaccount the positive and negative effects (e.g., insensitivity to painmay make it more difficult for the subject to notice when they hurtthemselves so they are more likely to injure themselves and worseninjuries that have occurred) of the gene and not affect the subject'sreadiness score, and the machine learning model 134 may take intoaccount only the negative effects and decrease the subject's readinessscore.

The risk score 144 may be calculated in a similar manner as describedabove with respect the readiness score 142. The machine learning models130, 132, and 134 may way various factors differently when calculating arisk score. The risk score 144 may be an average of each of the riskscores calculated by the machine learning models 130, 132, and 134, ormay be an average of the risk scores determined, e.g., by the scoringand prediction module 114 shown in FIGS. 1A-1C, based on the output ofthe machine learning models 130, 132, and 134.

There may be situations where a subject receives a relatively highreadiness score (a positive indication) but receives a relatively highrisk score (a negative indication). For example, the machine learningmodels 130, 132, and/or 134 may determine that the risk score of asubject should be increased, despite the subject currently performingwell, if their performance suddenly changes (e.g., quick increase ordecrease in the subject's performance), if their performance remainsstagnant for a prolonged period of time (e.g., multiple weeks ormonths), if their self-reported data indicates that they areexperiencing high levels of pain and/or continue to experience highlevels of pain, if their self-reported data indicates that they often oralways feel bad after completing a program challenge, if their genomicsdata indicates that they are predisposed to certain, problematic medicalconditions or are unlikely to perform well in certain scenarios, and/orif third-party reported data (e.g., from a coach, caregiver, potentialemployer, employer, supervisor, or the like) conflicts with obtainedsensor data and/or self-reported data.

In some implementations, the scoring and prediction module 114 shown inFIGS. 1A-1C calculates the readiness score 142 based on the output ofthe machine learning models 130, 132, and 134. Various factors mayaffect the subject's readiness score, such as the subject's performancetrend (e.g., whether the trend is positive or negative, whether thetrend is positively accelerating or negatively accelerating, etc.), theperformance of the subject on their program's challenges, how manychallenges the subject has completed or failed to complete, thepercentage of challenges the subject has completed or failed tocomplete, the genomics data of the subject, reported third-partyevaluation data (e.g., provided by a coach, caregiver, employer,potential employer, supervisor, etc.), self-reported data, or the like.

In some implementations, the scoring and prediction module 114 shown inFIGS. 1A-1C calculates the risk score 144 based on the output of themachine learning models 130, 132, and 134. Various factors may affectthe subject's risk score, such as the subject's performance trend (e.g.,too quick of performance increase may indicate a higher risk as well astoo slow of performance increase), how many challenges the subject hascompleted or failed to complete, the percentage of challenges thesubject has completed or failed to complete, the genomics data of thesubject, reported third-party evaluation data (e.g., provided by acoach, caregiver, employer, potential employer, supervisor, etc.),self-reported data, or the like.

In some implementations, the scoring and prediction module 114 shown inFIGS. 1A-1C includes the machine learning models 130, 132, and 134.Similarly, the scoring and prediction module 114 may include theintervention model(s) 136.

In some implementations, the output 140 may be presented on a userinterface. For example, the output 140 may be presented on a userinterface of a device of the corresponding subject. Similarly, theoutput 140 may be presented on a user interface of a device of a coach,counselor, caregiver, care provider, specialist, employer, or potentialemployer.

The output of the machine learning models 130, 132, and 134 may beprovided to one or more intervention models 136. The interventionmodel(s) 136 may include one or more machine learning models. Theintervention model(s) 136 may use the output of one or more of themachine learning models 130, 132, and 134 to generate and/or selectrecommendations or interventions for the subject.

The intervention model(s) 136 may also receive the subject data 116 asinput. When the intervention model(s) 136 receives the subject data 116as input, the intervention model 136 may use the subject data 116 toassist in generating and/or selecting recommendations or interventionsfor the subject.

In generating and/or selecting recommendations or interventions for thesubject, the intervention model(s) 136 may estimate an impact thatvarious recommendations or interventions will have on the likelihood ofthe subject meeting the readiness criteria, the estimated time it willtake for the subject to meet the readiness criteria, the likelihood ofthe subject reaching a certain goal or task, the estimated time it willtake for the subject to reach a certain goal or task, etc.

In estimating an impact that various candidate actions or interventionswill have, the intervention model(s) 136 may generate an impact scorefor every intervention in a list of interventions. The list ofinterventions may be particular to circumstances of the subject. Forexample, there may be a set of potential interventions corresponding tophysical therapy (e.g., if the subject is in physical therapy), theremay be a set of potential interventions corresponding to medicaltreatment in general, or to particular ailments or surgeries (e.g., ifthe subject is a patient), there may be a set of potential interventionscorresponding to employee training or evaluation (e.g., if the subjectis a potential employee or is an employee), there may be a set ofpotential interventions corresponding to consumers (e.g., if the subjectis a consumer), there may be a set of potential interventionscorresponding to physical fitness requirements or military requirements(e.g., if the subject is a recruit), or the like. Military requirementsmay include, for example, the criteria and/or passing score(s) for thearmy combat fitness test (ACFT), the armed services vocational aptitudebattery (ASVAB), and/or the armed forces classification test (AFCT).

The intervention model(s) 136 may determine, for example, based on thesubject data 116 whether the subject is a consumer, recruit, employee,potential employee, patient, or trainee. The intervention model(s) 136may proceed to (i) access one or more sets of potential interventionsbased on the whether the subject is a consumer, recruit, employee,potential employee, patient, or trainee, (ii) determine impact score(s)for one or more of the interventions (e.g., each of the interventionswithin each of the accessed sets of potential interventions) within theselected one or more sets of potential interventions, and, based on theimpact scores, recommend one or more the interventions (e.g., to acoach, caregiver) based on the impact scores. For example, theintervention model(s) 136 may determine based on the subject data 116that the subject is an Army recruit. Based on this information, theintervention model(s) 136 may obtain a set of potential interventionsfor physical fitness requirements and/or military requirements, and maydetermine an impact score for each of the interventions in the set(s) ofpotential interventions based on, for example, the subject's currentreadiness score 142, their risk score 144, their previous readinessscore(s), and/or their readiness score trend.

The intervention model(s) 136 may also take into consideration priorinterventions and how those interventions coincided with the subject'sreadiness scores, readiness score trend, and/or risks. As an example, ifa prior intervention making the subject's program more difficultcoincided with the subject receiving a lower readiness score(s) and/or aworsening of the subject's readiness score trend, then the interventionmodel(s) 136 may calculate an impact score for interventions that wouldmake the subject's program more challenging that is lower compared toimpact scores for interventions that would not make the subject'sprogram more challenging (e.g., would make the subject's program lesschallenging).

In some implementations, in estimating an impact that variousrecommendations or interventions will have, the intervention model 136may combine two or more interventions and estimate an impact thecombination(s) will have.

As shown, the intervention model 136 generates output 146. The output146 may include an indication of recommendations or interventions forthe subject. For example, the output 146 may include impact scores foreach of various recommendations or interventions. The scoring andprediction module 114 may use the output 146 to recommend one or more ofthe interventions (e.g., the three that have the highest beneficialimpact score). This recommendation may be sent to a device of thesubject, who may select one or more of the interventions. Thissuggestion may be sent to a device of a coach, counselor, caregiver,care provider, specialist, employer, potential employer, or supervisorcorresponding to the subject. The coach, counselor, caregiver, careprovider, specialist, employer, potential employer, supervisor mayproceed to select one or more of the interventions. A selection of oneof the interventions may result in change in the subject's program,program(s), challenge(s), goal(s), etc.

The scoring and prediction module 114 may use the output 146 toautomatically effect a change in the subject's program, program(s),challenge(s), goal(s), etc.

In some implementations, the scoring and prediction module 114 does notinclude and/or leverage the intervention model 136. The output of themachine learning models 130, 132, and 134 may be provided to a coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, or supervisor of the corresponding subject. The coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, or supervisor may then select, e.g., through a mobileapplication or website, one or more recommendations or interventions toprovide to the subject.

In some implementations, the scoring and/or predications take placelocally on one or more client devices, e.g., the client device 204 shownin FIG. 2A, the client device 304 shown in FIG. 3, or the personalmanagement device 610 shown in FIG. 6. Each of the one or more clientdevices may be, for example, a computing device such as a smart phone, alaptop computer, a desktop computer, a tablet, or the like. The one ormore client devices may belong to the subjects 102 a, 102 b, and/or 102c. As an example, the scoring and prediction module 114 may be locatedthe one or more client devices.

In some implementations, the subjects 102 a, 102 b, and/or 102 c use theone or more client devices to provide the self-reported information 150.For example, the subjects 102 a, 102 b, and/or 102 c may each provideprogram feedback through a mobile app on their respective clientdevices.

In some implementations, the one or more client devices are used tocollect and send sensor data. For example, the client devices mayinclude a GPS unit that can collect data indicating exercise durationand/or distance, a heart rate monitor indicating the respectivesubject's heart rate, a display unit to display one or more tests forthe respective subject to take and that can receive touch input, or thelike.

Models can be used to generate other types of predictions also, such ashow long readiness of the subject will be maintained after it isachieved, whether the subject will maintain readiness for at least aminimum amount of time or until a particular a time, an extent thatdifferent candidate actions will improve the maintenance of readinessfor the subject, and so on.

The system 100 is applicable to research efforts, as a tool to assistresearchers and facilitate scientific discovery. In this case, thesystem 100 may be leveraged to benefit researchers in designing,monitoring, and enhancing a study such as a clinical trial, a cohortstudy, or other research endeavor. For example, the subject whosereadiness is monitored may be a research study that is ongoing or isunder development. For example, the computer system 110 may be used topredict the likelihood completion of the study by individualparticipants (e.g., compliance with study requirements or adherence to aplan). As another example, the computer system 110 may be used topredict the likelihood that a study having certain parameters will toreach a desired outcome or progress to a desired level (e.g., have aminimum number of participants reporting a target level of data for atarget amount of time, or acquire an amount of data to provide astatistically significant result, etc.). The computer system 110 may beused to predict whether a study specification and/or proposed cohortwill likely achieve study objectives. In other words, the readinesscriteria for a research study can be, among other options, a readinessof a study to acquire a desired set of data or a readiness to be able toanswer a research question. For example, the models can be used topredict when parameters for a study being designed and/or composition ofa cohort being defined are ready so that, if used, are predicted toachieve at least a minimum level of confidence that the studyrequirements will be successfully completed by at least a minimum numberof participants.

The predictions of the computer system 110 can be used to iterativelydesign or improve a research study. As the study design is changed,e.g., as adjustments are made to study parameters (e.g., identity ofparticipants, number of participants, data types being measured,requirements of study participants, information gathering techniques,and so on) the computer system 110 can give updated predictions of thereadiness of the study design to reach target metrics (e.g., percentageof participants complying with study requirements, an amount of timeneeded to gather the types of information needed, a likelihood ofachieving a particular type of result, etc.). These predictions can bebased on examples of other studies, e.g., parameters used for thosestudies, readiness or success criteria for those studies, theprogression of the studies over time, and the ultimate outcomes for thestudies. The computer system 110 can analyze the examples to determinerelationships between different factors and outcomes and/or can usemachine learning to train models that implicitly learn relationshipsfrom the data. The computer system 110 can be used to providingprediction data to a user, to show predicted measures of readiness foreach different update to or version of a study design (e.g., showingchanging completion likelihoods for different sets of study parameters).The computer system 110 can also be used to provide recommendations ofchanges to the study that may improve the readiness of the study designfor implementation with high likelihood of success. This may includerecommending actions to change study parameters (e.g., changingcomposition of a cohort, data collection techniques, study duration,etc.) based on the trends, progressions, and outcomes of other examplestudies for which data is included in a database. One or more models mayadditionally or alternatively be trained to evaluate the potentialimpact of different changes on future success of the study. Based onthese evaluations, the computer system 110 can provide a score and/or ameasure of potential benefit for each of different candidate actions,along with user interface controls to implement those candidate actionsand change the study parameters.

As another example, for a research study in progress, the computersystem 110 can use models that assess the progress of the study, andgive updated predictions over time. As additional data for the study istracked and monitored by the computer system 110, the computer system110 can give updated predictions for the study, e.g., that the predictedtime of obtaining a complete data set may be later than earlierpredicted, or that successfully completion has become more likely givenrecent trends in participants' actions. Along with these predictions,the computer system 110 can provide recommended actions to improve ormaintain the readiness of the study to achieve predetermined objectivesexpressed as readiness criteria. The computer system 110 can be used toassess the progress of a study and evaluate whether the study isprogressing toward successful completion, as defined by readinesscriteria for the study. The computer system 110 can also evaluatecandidate actions for the research study, such as changing studyparameters (e.g., information gathered, techniques for gatheringinformation, types of interactions, questions addressed, study duration,number and identity of participants, and so on). The computer system 110can assign scores to the candidate actions to indicate how likely eachcandidate action to improve the progression of the study towardsuccessful completion. As another example, the candidate actionsoptionally do not changing fundamental study parameters, but may alterinteraction with participants to adjust factors such as the frequency ortype of reminders to study participants. These factors can bepotentially customized for each individual participant, based on exampledata in the database about how similar types of participants haveresponded in other studies.

FIG. 2A is a diagram that illustrates an example process 200 a forpredicting readiness. The process 200 a may be performed by the system100 shown in FIG. 1A. As shown, part of the process 200 a is performedon the computer system 110.

The process 200 a includes generating and collecting data (210). Duringdata generation and collection, data of the subject 202 can be obtainedfrom multiple collections sources. The collection sources may includeone or more sensing devices (e.g., the sensing devices 104 a-104 c shownin FIG. 1A), a client device 204 of the subject 202, a third-partydevice 208, and/or one or more external systems (e.g., an EMR system).For example, some of the data, such as information self-reported by thesubject 202, may be obtained from the client device 204 of the subject202. Prior to data generation and collection, the subject 202 may havebeen provided an initial program, e.g., through one or morerecommendations and interventions, by a third-party 206. The third-party206 may be, for example, coach, counselor, caregiver, care provider,specialist, employer, potential employer, supervisor, or the like. Thisinitial program may not be personalized, e.g., because it is not basedon data collected from the subject 202.

The process 200 a includes processing the data (212). Processing thedata may include extracting performance information from the data thatindicates a performance of the subject 202. The performance informationmay indicate a physiological performance of the subject 202 and/or apsychological performance of the subject 202.

The process 200 a includes aggregating the data (214). Aggregating thedata may include organizing the data based on the subject 202, the typeof subject that the subject 202 is (e.g., potential employee, recruit,patient, etc.), the time from when the data was collected, the type ofperformance information within the data, etc. The data may be aggregatedby the data aggregation module 112 shown in FIGS. 1A-1C.

The process 200 a optionally includes assessing the aggregated datausing a subject model for a subject benchmark or readiness criteria(216). Assessing the aggregated data using a subject model may includecomparing the aggregated data with readiness criteria. The assessmentmay indicate whether the subject 202 currently meets the subjectbenchmark/readiness criteria. Assessing the aggregated data may beperformed by the scoring and prediction module 114 shown in FIGS. 1A-1C.

The process 200 a optionally includes assessing the aggregated datausing a group model for a group benchmark or readiness criteria (218),e.g., when the subject 202 belongs to a group or a team of subjects thatare being evaluated. Assessing the aggregated data using a group modelmay include comparing the aggregated data with the group readinesscriteria. The assessment may indicate whether the subject 202 currentlymeets the group benchmark/group readiness criteria. Assessing theaggregated data may be performed by the scoring and prediction module114 shown in FIGS. 1A-1C.

The process 200 a includes, based on the assessment(s), making one ormore decisions and/or scoring the subject (220). For example, withrespect to FIG. 1B, the scoring and prediction module 114 may determinea readiness score for the subject 202, a risk score for the subject 202,a performance trend for the subject 202, a determination as to whetherthe subject 202 is likely to meet readiness criteria (e.g., a particularreadiness score), a determination as to when the subject 202 is likelyto meet readiness criteria, or the like. The scoring and predictionmodule 114 may make these determinations using the machine learningmodels 130, 132, and/or 134. As another example, the scoring andprediction module 114 may determine one or more interventions for thesubject 202's program in order to improve the likelihood of the subject202 achieving readiness (e.g., by meeting the readiness criteria) or byreducing the time until the subject 202 is likely to achieve readiness.The scoring and prediction module 114 may determine these one or moreinterventions using the intervention model(s) 136.

The process 200 a includes training one or more machine learning models(222). The one or more machine learning models may be trained using thecurrent information such as the one or more decisions and/or scorescorresponding to the subject 202, and/or using past informationcorresponding to the subject 202 or other subjects. With respect to FIG.1B, the one or more machine learning models may include the machinelearning models 130, 132, and/or 134, and/or the intervention model(s)136. For example, the generated readiness score(s) for the subject 202may be used to train the machine learning models 130, 132, or 134 and/orthe intervention model(s) 136 shown in FIG. 1B.

The one or more decisions and/or scores corresponding to the subject 202are sent to the third-party device 208 of the third-party 206. Forexample, the third-party 206 may be a coach of the subject 202. Thethird-party 206 may receive an indication of the subject 202's currentreadiness, an indication as to whether the subject 202 is likely toachieve readiness with their current training program, an indication ofwhen the subject 202 is likely to achieve readiness, and/or one or morerecommended interventions for the subject 202's training program.

The process 200 a includes providing recommendations and/orinterventions to the subject (224). For example, the third-party 206 maygenerate one or more recommendations and/or interventions for thesubject 202 based on the one or more decisions and/or scorescorresponding to the subject 202. As another example, the third-party206 may select one or more of the interventions contained within the oneor more decisions and/or scores corresponding to the subject 202.Specifically, the recommendations and/or interventions may be a subsetof recommendations and/or interventions suggested by one or more machinelearning models of the computer system 110. The recommendations and/orinterventions can be sent from the third-party device 208 to the subject202's client device 204, e.g., through the network 160 shown in FIG. 1A.

FIG. 2B is a diagram that illustrates an example process 200 b forpredicting readiness. The process 200 b may be performed by the system100 shown in FIG. 1A.

The process 200 b includes defining performance readiness (230).Defining performance readiness may include defining one or moreperformance criteria. Performance criteria may include physiologicalbenchmarks (e.g., physical performance requirements) and/orpsychological benchmarks (e.g., performance on a particular examination,performance on an IQ test, etc.). Performance readiness may be definedby a third-party such as, for example, a coach, a counselor, acaregiver, a care provider, a specialist, an employer, a potentialemployer, a supervisor, or the like. Performance readiness may bedefined for a particular subject, a particular group of subjects, or aparticular team of subjects in a group of subjects.

The process 200 b includes collecting measurement data from a subject ora group of subjects (232). With respect to FIG. 1B, the measurement datamy include sensor data 106 a, 106 b, or 106 c collected using sensingdevices 104 a, 104 b, or 104 c, respectively.

The process 200 b includes selecting at least one model forinstantaneous comparison (234). The model may be a machine learningmodel, such as the machine learning model 130, 132, and/or 134 shown inFIG. 1B. The at least one model selected may include a model thatanalyzes reactionary results based on the measurement data withoutcomparing it to past measurement data. For example, the at least onemodel may analyze the measurement data along with othercontemporaneously collected data. The at least one model may receive themeasurement data as it is collected. For example, the at least one modelmay analyze the measurement data is it is received, e.g., in real-timeor substantially real-time. The model(s) selected may be based on thetype of subject(s), the readiness criteria for the subject(s), thethird-party that set the readiness criteria, or the like.

The process 200 b includes selecting at least one model for longitudinalcomparison (236). The at least one model may be a machine learningmodel, such as the machine learning model 130, 132, and/or 134 shown inFIG. 1B. The at least one model selected may include a model thatanalyzes historical trends of data, such as historical trends ofmeasurement data of the subject, the group of subjects, or a team ofsubjects in the group of subjects. The model(s) selected may be based onthe type of subject(s), the readiness criteria for the subject(s), thethird-party that set the readiness criteria, or the like.

The process 200 b includes scoring performance readiness to a subject ora group of subjects (238). With respect to FIGS. 1A-1C, scoringperformance readiness to a subject or a group of subjects may beperformed by the scoring and prediction module 114 on the computersystem 110. Scoring performance readiness to a subject or a group ofsubjects may be performed using the selected at least one model forinstantaneous comparison, and the at least one model for longitudinalcomparison.

The process 200 b optionally includes predicting a completion readinesstime (240). Completion readiness time can include the time it takes fora subject, a group of subjects, or team of subjects in a group ofsubjects to achieve readiness. For example, a completion readiness timefor a subject can be the time for the subject to achieve a particularreadiness score. In the case of group of subjects or a team of subjects,a completion readiness time may include the time for the group or teamto achieve a particular, average readiness score, or a time for eachsubject in the group or team to achieve a particular readiness score.

The process 200 b optionally includes predicting a future readinessprobability (242). Future readiness probability can include a likelihoodthat a subject, a group of subjects, or team of subjects in a group ofsubjects will achieve readiness, or will achieve readiness by a certaintime, e.g., by the end of a scheduled program. For example, a futurereadiness probability for a subject can be the likelihood of the subjectachieving a particular readiness score. In the case of group of subjectsor a team of subjects, a future readiness probability may include thelikelihood for the group or team to achieve a particular, averagereadiness score, or a likelihood for each subject in the group or teamto achieve a particular readiness score.

The process 200 b optionally includes predicting an altered coursesuccess rate (244). An altered course success rate may be, for example,a likelihood that a subject, a group of subjects, or team of subjects ina group of subjects will achieve readiness, or will achieve readiness bya certain time, if a change to their respective program(s) is made. Thechange may include one or more interventions, e.g., outputted by theintervention model(s) 136 shown in FIG. 1C.

The process 200 b includes delivering intervention recommendations(246). With respect to FIG. 1C, delivering intervention recommendationsmay be performed by the intervention model(s) 136 shown in FIG. 1C. Theintervention recommendations may be provided to one or more clientdevices of a subject, a group of subjects, or a team of subjects in agroup of subjects. The intervention recommendations may be provided toone or more third-party devices, such as mobile devices belonging to acoach or caregiver of the corresponding subject, group of subjects, orteam of subjects.

The process 200 b includes executing selected interventions to a subjector a group of subjects (248). Executing selected interventions to asubject or group of subjects may include changing the program of subjector each of the programs of a group or team of subjects based on thepredictions. The program(s) may be changed automatically, e.g., by thecomputer system 110 shown in FIGS. 1A-2A, and 3. The program(s) may bechanged manually by, for example, one or more third-parties who haveaccessed or been provided the predictions.

FIG. 3 is a diagram that illustrates an example process 300 forassessing a subject 302. The process 300 may be performed using thesystem 100 shown in FIG. 1A. Specifically, the process 300 may beperformed on the computer system 110 after receiving self-reportedinformation 150, sensor data 106, third-party input 152, electronichealth records 154, and/or genomics data 156. As shown, theself-reported information 150 may be provided by a subject 302 through aclient device 304. The sensor data 106 may be obtained through clientdevice 304 or through, for example, one or more sensing devices. Theelectronic health records 154 may be obtained through client device 304or through, for example, an external EMR system. The genomics data 156may be obtained through client device 304 or through, for example, anexternal EMR system. The third-party input 152 may be provided by athird-party 306 through a third-party device 308.

In some implementations, the client device 304 is the client device 204shown in FIG. 2A. In some implementations, the subject 302 is thesubject 202 shown in FIG. 2A. In some implementations, the third-partydevice 308 is the third-party device 208 shown in FIG. 2A. In someimplementations, the third-party 306 is the third-party 206 shown inFIG. 2A.

In the process 300, the subject 202 may be assessed based on genomicsand metabolic analysis factors, and a tailored training program 332 maybe produced for the subject 202 based on the assessment.

As shown, in the process 300, a genomics assessment 310 is performed.The genomics assessment 310 determines performance contributors 312 andphysical limitations or vulnerabilities 314. The performancecontributors 312 and the physical limitations or vulnerabilities 314 canbe used to generate or update risk factors 316. The genomics assessment310 can provide both positive and negative scoring into the tailoredtraining program 332, e.g., through the risk factors 316. As an example,the table 400 shown in FIG. 4 provides an example of results from one ormore genomics assessments that may be performed in the process 300.

In the process 300, a metabolic assessment 320 is performed. Themetabolic assessment 320 assessment compares historical contributions322 and the subject 302's current situation 324 to provide a deeperanalysis into their given body development trends. The historicalcontributions 322 and the subject 302's current situation 324 can beused to generate or update scoring factors 326. The metabolic assessment320 can influence the tailored training program 332, e.g., through thescoring factors 326. As an example, the table 500 shown in FIG. 5provides an example of results from one or more metabolic assessmentsthat may be performed in the process 300.

The results of the genomics assessment 310 and the metabolic assessment320 may be combined to provide a complete human performance assessment330 for the subject 302. The human performance assessment 330, the riskfactors 316, and the scoring factors 326 may be used to develop and/orupdate the tailored training program 332 for the subject 302.

Machine learning models and/or artificial intelligence tools may be usedin generating or updating the risk factors 316 and/or the scoringfactors 326. The models and/or tools could be used to optimize thetailored training program 332 overtime, for example, based on thesuccess and/or failure of other implemented tailored training programsin the community of subjects.

The tailored training program 332 may be optionally provided to thesubject 302 through their client device 304. The tailored trainingprogram 332 may be optionally provided to the third-party 306 throughtheir third-party device 306.

FIG. 4 is a diagram that illustrates an example table 400. The table 400depicts the results of an example genomics assessment (e.g., thegenomics assessment 310 shown in FIG. 3). As shown, the table 400depicts how particular genes may correspond to conditional positiveand/or negative effects positive. These effects, or a net effects, maybe implemented as positive and/or negative scoring into the tailoredtraining program 332 shown in FIG. 3.

As shown in the table 400, the genes may be labeled as a strengthrelated (“S”), cardiovascular related (“C”), respiratory related (“R”),brain related (“B”), or an undesirable (“U”) gene. The S labeled genescan impact the physiological strength and endurance (e.g., may indicatethat a corresponding subject has a higher likelihood of successfullyenduring more intense physiological related training). The C labeledgenes can impact blood flow and circulatory aspects of the human body.The R labeled genes can impact air inhalation, exhalation and thedelivery of oxygen throughout the body. The B labeled genes can impactcognitive functions and mental capacity, ability to replenish awarenessand alertness.

Each of the genes shown in the table 400 labeled have conditionalpositive effects that can be applied as beneficial factors. Some of thegenes shown in the table 400 also has conditional negative effects,which is indicated as (U) or undesirable outcomes, vulnerabilities, andrisks. The conditional positive effects may provide an increase to thesubject's score by the depicted amount if it the correspondingbeneficial effect is relevant to the subject's evaluation. Similarly,the conditional negative effects may result in a decrease to thesubject's score by the depicted amount if the corresponding detrimentaleffect is relevant to the subject's revaluation. As an example, the geneLRP5 has a conditional positive effect due to the strength relatedbenefits that extra strong bones provide, but also has conditionalnegative effect due to the extra strong bones resulting in a lowerbuoyancy. The lower buoyancy results in conditional negative effects asit may make it more difficult for the subject to excel at swimmingwithout specialized training. However, if swimming were not part of orrelated to the subject's evaluation, then the subject might not havetheir readiness score detrimentally affected from having the LRP5 gene.

As another example, an undesirable risk of the insensitivity to painbenefit associated with strength related genes SCN9A and FAAH-OUT isthat these genes also may provide unnoticed harm. This may mean that thesubject can train harder but is more likely to hurt themselves in theprocess (e.g., could result in an unnoticed blood clot if the subjectstands for long periods, or fractures that can worsen without notice).

An undesirable vulnerability of being resistant to Malaria for thecardiovascular related gene HBB is that these genes also can be linkedto a sickle cell trait. The sickle cell trait creates a risk duringtraining that may lead to sudden death, and need further monitoring andprecision tailoring of human performance training.

In some implementations, with respect to FIGS. 1A-1C, the computersystem 110 receives the genomics assessment represented as the table 400from an external database such as, for example, an EMR system. Forexample, the computer system 110 may receive the genomics assessmentrepresented as the table 400 as all or part of the electronic healthrecords 154 or the genomics data 156.

The computer system 110 may provide the genomics assessment representedas the table 400 to the data aggregation module 112. Accordingly, thegenomics assessment may be used by the scoring and prediction module indetermining a readiness score for the corresponding subject and/or oneor more predictions for the corresponding subject. Specifically, thegenomics assessment represented as the table 400 may be included in thesubject data 116 and may be provided to the machine learning models 130,132, and 134, and/or provided to the intervention model(s) 136.

In some implementations, the machine learning models 130, 132, and/or134 take into account the genomics of the subject as indicated bygenomics assessment represented as the table 400 when calculating areadiness score and/or a risk score for the subject. For example, one ormore of the machine learning models 130, 132, and 134 may calculate ahigher risk score for the corresponding subject if they have the genesSCN9A and/or FAAH-OUT as these genes indicate an insensitivity to pain.If one or more of these genes are present, the subject would be morelikely to injure themselves while completing program tasks or exercises.Accordingly, the machine learning models 130, 132, and/or 134 maydetermine a higher risk score for the subject than for other, similarperforming, subjects who do not have the SCN9A and/or FAAH-OUT genes.

However, depending on the readiness criteria, the machine learningmodels 130, 132, and/or 134 may calculate a higher readiness score forsubject having the genes SCN9A and/or FAAH-OUT. For example, if thereadiness criteria requires that the subject must perform the well in abattle, the insensitivity to pain that corresponds with those genes maybe viewed as an advantage. Accordingly, the machine learning models 130,132, and/or 134 may calculate a higher readiness score for the subjectthan for other, similar performing, subjects who do not have the SCN9Aand/or FAAH-OUT genes.

In some implementations, the intervention model(s) 136 shown in FIG. 1Cuse(s) information in the genomics assessment represented as the table400 to calculate a prediction (e.g., a percent decrease in time to reacha particular goal or readiness score, or a percent increase in time toreach a particular goal or readiness score), or to generate arecommendation including one or more interventions. For example, if thesubject has the LRP5 gene that indicates a higher bone density, theintervention model(s) 136 would not recommend an intervention thatincludes adding a swimming regimen to the subject's program since thesubject has a low buoyancy due to their higher bone density as indicatedby the LRP5 gene.

FIG. 5 is a diagram that illustrates an example table 500. The table 500depicts the results of an example physical or metabolic assessment,e.g., the metabolic assessment 320 shown in FIG. 3. As shown, the table500 depicts examples of metabolic assessments. The metabolic assessmentsmay include a minimum calculated value, and a maximum calculated valuefor a particular test. The metabolic assessments may also include one ormore targets or goals that the subject should eventually reach or beattempting to reach. The targets may include a first target thatcorresponds with initial goal that the subject should try to reach(e.g., a goal that is made part of the subject's program and is set forthe subject to attempt to reach in four weeks), and a second target thatcorresponds with a second or final goal that the subject should try toreach (e.g., a goal that is made part of the subject's program and thatis set for the subject to attempt to reach by the conclusion of theprogram in ten weeks). The table 500 also provides equations used incalculating the various values.

Multiple metabolic assessments may be performed on a subject over time.Previous assessments may be compared to the most recent assessment ormore recent assessments, and the comparison(s) may be used indetermining a readiness score for the subject. As an example, detailslike VO2Max along with regular exercise targets and comparison to thosetargets and previous scoring provides a degree of change based on thesubject and is an indicator into their tailored human performancetraining.

There are multiple ways to calculate VO2Max. One such way is to measurethe maximum oxygen consumption in liters over a minute and divide it bybody weight in kilograms. An alternative to this, and one suggested bythe table is to take the maximum heart rate (MHR) and divide it by theresting heart rate (RHR). Collection of this information regularly by awearable such as a chest strap, ear sensor, wrist sensor, finger sensor,or equipment based sensor such as those found on a treadmill, providethe ability to derive this information over time from a consumer withinformation captured throughout and prior to any human performancetraining.

As shown, the metabolic or physical criteria may include deadlifts,power throws, pushups, sprints, pull-ups, and a two-mile run. Each ofthe metabolic or physical criteria may include a minimum value (“MinX”), e.g., an average value of the lowest 25%, 15%, 10%, or 5% ofperformers in the corresponding exercise. Each of the metabolic orphysical criteria may include a maximum value (“Max X”), e.g., anaverage value of the highest 25%, 15%, 10%, or 5% of performers in thecorresponding exercise.

In some implementations, the minimum values (“Min X”) correspond tominimum acceptable levels of performance at the beginning of testing(e.g., at the onset of the subject's program). Whereas, the maximumvalues (“Max X”) correspond to minimum acceptable levels of performanceat the end of testing (e.g., at the conclusion of the subject'sprogram).

In some implementations, the minimum values (“Min X”) and the maximumvalues (“Max X”) define a range of acceptable performance measures at apoint in the testing of the subject. For example, the values may definea range of acceptable performance measures at the beginning of thesubject's program, at a point in time after the subject started theprogram (e.g., two weeks in, four weeks in, three months in, etc.), orat the conclusion of the subject's program.

The metabolic or physical criteria found in the table 500 maycorresponding to particular requirements accessed from a database, e.g.,by the computer system 110 shown in FIGS. 1A-1C, or set by a coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, or supervisor. As an example, the metabolic or physicalcriteria found in the table 500 may correspond to physical requirementsset by a Military branch recruiting subjects. These requirements mayinclude, for example, the criteria and/or passing score(s) for the armycombat fitness test (ACFT).

In some implementations, the first target (“Target 1”) and/or the secondtarget (“Target 2”) for each of the metabolic or physical criteria maybe preselected.

In some implementations, the first target (“Target 1”) and/or the secondtarget (“Target 2”) are dynamically selected. For example, with respectto FIGS. 1A-1C, the targets may be selected based on the subject'scurrent readiness score, current risk score, past readiness score(s),and/or trend of readiness scores, e.g., by computer system 110 afterreferring to a lookup table, or by the computer system 110 afteranalyzing output of the intervention model(s) 136. As another example,with respect to FIGS. 1A-1C, the targets may be set by a coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, or supervisor, e.g., based on the subject data 116 afterreceiving the subject data 116 from the computer system 110, or based onoutput from the intervention model(s) 136 after receiving the outputfrom the computer system 110. The coach, counselor, caregiver, careprovider, specialist, employer, potential employer, or supervisor mayset the targets as part of the third-party input 152.

In some implementations, the first target (“Target 1”) and/or the secondtarget (“Target 2”) are dynamically updated, e.g., based on newassessment data for the corresponding subject (e.g., new calculatedreadiness score, risk score, performance data, etc.), and/or based onnew self-reported data from the corresponding subject or from a coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, or supervisor (e.g., feedback indicating the subject's painlevel, or feelings).

FIG. 6 is a diagram that illustrates an example system 600 forgenerating precision predictions of readiness. As shown, the system 600includes deployment dedicated sensors 602 having a data store 604, apersonal management device 610, and a server 620. The personalmanagement device 610 may be a computing device such as a smart phone, alaptop computer, a desktop computer, etc. In some implementations, theserver 620 is the computer system 110 shown in FIGS. 1A-1C. In someimplementations, the deployment dedicated sensors 602 are one or more ofthe sensing devices 104 a-104 c shown in FIG. 1A.

On the client deployment side, the personal management device 610includes a personalized coaching intervention user interface 612, apersonalized outline and content 614, and data acquisition module 616.The personalized coaching intervention user interface 612 can be aninterface that describes the client side action and activities,measurements, and the modifications to any and all programs. Thepersonalized outline and content 614 can be information that the subjectis required to adhere to or complete against in order to stay incompliance with the program and will effectively be measured against.For example, the personalized outline and content 614 may includecriteria, goals, challenges, etc. that a subject has been tasked with.This initially begins as a baseline outline and content but is furthermanaged by the personal management device 610 to acquire and reconfigurethe experience and training that the subject is required to complete.The personal management device 610 may serve as the manager of thepersonalized outline and content 614, e.g., by making changes to thesubject outline and content 626 a located on the server 620.

The data acquisition module 616 may receive collected data from the datastore 604 of the deployment dedicated sensors 602. The collected datamay include self-reported information, or data from one or more sensorsor sensor networks. Sensors may exist within the personal managementdevice 610 or as an external connection or set of connections availableto the personal management device 610.

The server 620 includes a management system 622 having a data store 632.The data in the data store 632 is managed by the management system 622and is specific to the management system 622 and the performance of themanagement system 622. For example, the data in the data store 632 maybe leveraged by the management system 622 to assist the managementsystem 622 in carrying out logical operations.

The server 620 also includes a coaching and community management system624 having a machine learning algorithm store 634, a machine learningtraining data store 636, a program data store 638, and data of multiplesubjects. As shown, the management system 624 includes a subject outlineand content 626 a and measurements 628 a corresponding to a firstsubject, and a subject outline and content 626 b and measurements 628 bcorresponding to a second subject. A subject may interact with themanagement system 624 through a web portal or a mobile applicationconnected to the resources. The management system 624 may allow asubject to access, for example, system settings, a status of subjects,data corresponding to subjects, data corresponding to groups or teams ofsubjects, classification data, etc. Through the management system 624, asubject may make changes to the program of one or more subjects, and tothe algorithms that are required to manage the projected needs of eithercompliance or readiness for one or more subjects reported by the system624.

The program data store 638 stores the core information related to theclasses in which adherence and readiness are scored. The program itselfdescribes various alternate pathways for varying subject levels based ontheir readiness. There can be many programs and many outlines andcontent describing relevant programs. The program data store 638 maystore programs that correspond to a type of assessment (e.g., subjectsare being evaluated for the same or similar position, undergoing therapyor treatment for the same or similar type of injury, undergoing the sameor similar type training, etc.) or to a type of subject (e.g., aconsumer, an employee, a potential employee, a trainee, etc.).

The machine learning algorithm store 634 stores one or more machinelearning algorithms or models for scoring criteria required to determinereadiness and predict future readiness for subjects. For example, theone or more machine learning algorithms or models may be used todetermine a readiness score for a particular subject, a risk score for aparticular subject, a performance trend for a particular subject, aconfidence of a particular subject achieving readiness or another goal,and/or an expected time for the particular subject to achieve readinessor another goal. The machine learning algorithm store 634 may include,for example, the machine learning models 130, 132, and 134, and/or theintervention model(s) 136 shown in FIG. 1C.

The machine learning training store 636 stores information required toprovide high accuracy and confidence in the scoring output, e.g., theoutput of the scoring and prediction module 114 shown in FIG. 1C. Themachine learning training store 636 may store past data of one or moresubjects, e.g., past sensor data, past self-reported data, pastthird-party data, past readiness scores, past performance trends, pastrisk scores, past program completion percentages, or the like. Themachine learning training store 636 may store one or more training setsdeveloped from past data of one or more subjects.

Each subject has their own personalized data storage and access that isutilized by the subject across multiple trainings inclusive of programswith outline and content.

The subject outline and content 626 a-626 b may be unique to aparticular subject based on their performance level and supports themanagement functions on the client side, e.g., may be modified based oncustom configurations made to the personalized outline and content 614on the personal management device 610. A subject's outline may be, forexample, their program. This program may be updated overtime, e.g.,through one or more interventions recommended by the interventionmodel(s) 136 shown in FIG. 1C. A subject's content may include, forexample, a current or past readiness score, performance trend, or riskscore for the subject. A subject's content may include, for example, aprediction such as whether the subject is expected to reach a particulargoal (e.g., a set readiness score by a certain time) or when the subjectis expected to reach a particular goal. The subject outline and content626 a may be accessed and modified by the personal management device610, resulting in the personalized outline and content 614. Afterwards,the server 620 may receive the personalized outline and content 614 andreplace the subject outline and content 626 a with the personalizedoutline and content 614 or may use the personalized outline and content614 to update the subject outline and content 626 a.

In some implementations, the subject outline and content 626 a-626 b isnot unique to a particular subject, e.g., at least initially. That is,the subject outline and content 626 a-626 b may start as a copy of aprogram within the program data store 638. As an example, the subjectoutline and content 626 a-626 b can be, or at least start, general to atype of assessment (e.g., subjects are being evaluated for the same orsimilar position, undergoing therapy or treatment for the same orsimilar type of injury, undergoing the same or similar type training,etc.) or to a type of subject (e.g., a consumer, an employee, apotential employee, a trainee, etc.). However, the subject outline andcontent 626 a-626 b may be updated overtime for the respective subject,e.g., based on respective subject's measurements 628 a or 628 b, and/orbased on the output of one or more of the machine learning models in thestore 634.

The measurements 628 a-628 b are the data recorded for each subject. Themeasurement 628 a and/or the measurement 628 b can each be analyzed fora subject or may be analyzed together, e.g., when the correspondingsubjects belong to a group or team. Similarly, measurement data can becompared based on the metadata describing the subject's characteristicswith other like-minded characterized subjects, for example, in verticaloccupations. The measurements 628 a-628 b may include sensor datacollected from the corresponding subjects. The measurements 628 a mayreceive and be updated with data provided by the data acquisition module616 of the personal management device 610.

In some implementations, the management system 622 provides themeasurements 628 a-628 b to one or more of the machine learning modelsin the store 634 in order to determine the corresponding subject'sreadiness score, the corresponding subject's performance trend, thecorresponding subject's risk score, and/or one or more predictionsinvolving the corresponding subject such as the likelihood of themreaching a predetermined goal.

In some implementations, the management system 622 uses the output of amachine learning model in the store 634 to update the subject outlineand content 626 a-626 b. For example, the output of a machine learningmodel in the store 634 may indicate that the corresponding subject is nolonger likely to reach a predetermined goal that is part of the subjectoutline and content 626 a and/or 626 b. As another example, the outputof a machine learning model in the store 634 may indicate one or moreinterventions that could be automatically (or manually) added to thecorresponding subject's program. Accordingly, the management system 622may automatically select one or more interventions and to update thesubject outline and content 626 a and/or 626 b.

FIG. 7 is a diagram that illustrates the data used by an example system700 for generating precision predictions of readiness. The system 700includes a subject measurement data store 702, a coaching curriculum andtraining data store 704, a community and public resources data store706, an application data store 708, and a system 710. The subjectmeasurement data store 702, coaching curriculum and training data store704, community and public resources data store 706, application datastore 708, and network 720 may form a data network. In someimplementations, the system 710 is the computer system 110 shown inFIGS. 1A-1C. In some implementations, the network 720 is the network 160shown in FIG. 1A.

In evaluating one or more subjects, the system 710 may access data fromthe subject measurement data store 702, coaching curriculum and trainingdata store 704, community and public resources data store 706, and/orapplication data store 708.

The system 710 may update data on the subject measurement data store702, coaching curriculum and training data store 704, community andpublic resources data store 706, and/or application data store 708,e.g., based on new sensor data, based on new self-reported data, basedon new assessments, etc.

In some implementations, the subject measurement data store 702,coaching curriculum and training data store 704, community and publicresources data store 706, and/or application data store 708 are part ofthe system 710, e.g., are found on local storage of the system 710.

The subject measurement data store 702 may include subject providedinformation, subject reported outcomes, subject sensor data,environment/exposures, passively sensed/behaviors, bioassay (blood,urine, saliva, etc.), genomics, and EHR/My Health. With respect to FIG.1C, the subject measurement data store 702 may include the self-reportedinformation 150, the electronic health records 154, and/or the genomicsdata 156. With respect to FIG. 1C, data within the subject measurementdata store 702 may be provided to the machine learning models 130, 132,and 134 and/or to the intervention model(s) 136 as the subject data 116.The machine learning models 130, 132, and/or 134 may use all or part ofthe data within the subject measurement data store 702 that correspondsto a particular subject in determining a readiness score for thesubject, determine a risk for the subject, determine a performance trendfor the subject, determine whether the subject is likely to reach apredetermined goal, determine a confidence of the subject reaching apredetermined goal, or determine an estimated time for the subject toreach the predetermined goal.

Similarly, the intervention model(s) 136 may use all or part of the datawithin the subject measurement data store 702 that corresponds to aparticular subject in determining one or more interventions for thesubject in order to, for example, improve the likelihood of the subjectreaching a predetermined goal, improve the readiness score of thesubject, lessen the risk of the subject, lessen the estimated time forthe subject to reach a predetermined goal, or to account from changedgoals for the subject (e.g., if a coach, employer, counselor, or otherof the subject updates or modifies the goals for the subject).

The coaching curriculum and training data store 704 may includeinterventions, omics, phenotyping, markers, education, programs/outlinesand content, and personalized machine learning. With respect to FIG. 1C,the coaching curriculum and training data store 704 may include data inthe third-party input 152. The interventions in the coaching curriculumand training data store 704 may be those outputted and/or recommended bythe intervention model(s) 136 shown in FIG. 1C. The program/outlines andcontent in the coaching curriculum and training data store 704 mayinclude the initial program assigned to corresponding subjects and/or acurrent program assigned to corresponding subjects, e.g., one that hasbeen updated over time based on the corresponding subjects' measurementsand/or the outputs of one or more machine learning models that receivethe corresponding subjects' measurements.

The community and public resources data store 706 may include groupdata, measurement guidance, research findings, SNPs, environmentsensors, standard level of care, and machine learning training. The datastored in the community and public resources data store 706 may includepast performance data of subjects, groups of subjects, and/or teams ofsubjects, e.g., that was used to create one or more training sets. Thedata in the community and public resources data store 706 may be togenerate one or more training sets or may be already organized into oneor more training sets. These training set may be used to train one ormore machine learning models, e.g., with respect to FIG. 1C, the machinelearning models 130, 132, and/or 134, and/or the intervention model(s)136.

The application data store 708 may assist with data preparation, datapersonalization, and data processing.

FIG. 8 is a diagram that illustrates an example scoring of subjects. Thescoring may be done by, for example, the computer system 110 shown inFIGS. 1A-1C. The scoring may be done by, for example, the system 710shown in FIG. 7.

As shown in FIG. 8, the system 100 has determined a readiness score foreach of twenty-seven subjects. The system 100 has also divided thetwenty-seven subjects into three teams 802 a, 804 a, and 806 a. Thefirst team 802 a has an average readiness score of 6.6. The second team804 a has an average readiness score of 6.3. The third team 806 a has anaverage readiness score of 5.4. As a whole, the group of twenty-sevensubjects has an average readiness score of 6.1, e.g., the averagereadiness among the three teams 802 a, 804 a, and 806 a.

In some implementations, the system 100 may improve the overall groupreadiness and/or the readiness of the each of the teams 802 a, 804 a,and 806 a by repositioning subjects. Repositioning the subjects mayinclude the system 100 removing one or more subjects from the group(e.g., those having a readiness score below a particular threshold), oradding one or more subjects to the group (e.g., those having a readinessscore at or above a particular threshold). Repositioning the subjectsmay additionally or alternatively include the system 100 moving one ormore subjects to a different team within the group. For example, asshown, the system 100 has removed the lowest scoring subjects, thosewith a readiness score of 4/10, from the group. In addition, the system100 has moved a subject having a score of 6/10 from the third team 806 ato the first team 802 a, and has moved a subject having a score of 8/10from the first team 802 a to the third team 806 a. This results in theaverage readiness score of the third team 806 b increasing to 6.2, andthe overall readiness score for the group increasing to 6.4.

In some implementations, the system 100 determines which subjects toreposition is based on the characteristics of the subject and/or theevaluation being performed. For example, the system 100 determines whichsubjects to reposition based on an occupation or training criteria, andwhether to expel or provide a degree of negative punishment.

In some implementations, the system 100 requires new scores of theoriginal twenty-seven subjects after they have completed respectiveinterventions. The new scores indicate that, on average, the subjectsincreased their readiness scores. As shown, the average readiness scorefor the first team 802 c increased to 7.0, the average readiness scoreof the second team 804 c increased to 7.2, the average readiness scoreof the third team 806 c increased to 6.8, and the overall readinessscore for the group increased to 7.0.

Through repositioning and/or the obtaining of new readiness scores, thesystem 100 can increase the average readiness scores of teams and of agroup. In addition, by repositioning subjects, the system 100 can formteams that are likely to be better balanced, e.g., that performsimilarly to one another, or are at least expected to perform similarlyto one another.

FIGS. 9A-9E are diagrams that illustrate example interfaces 900 a-900 eshowing assessment analytics and results. The interfaces 900 a-900 e maybe displayed on a computing device that received data from the computersystem 110. The interfaces 900 a-900 e may be displayed a device of auser assigned or entrusted with monitoring or assisting subjects toachieve and maintain readiness, such as on the third-party device 208 ofthe third party 206 (e.g., a coach, caregiver, employer, potentialemployer, supervisor, and the like). In some implementations, some orall of the interfaces 900 a-900 e can be provided for display by devicesof subjects whose progress is being monitored.

FIG. 9A depicts a user interface 900 a that provides a list view ofsubjects being assessed and analytical data corresponding to thosesubjects, such as program completion percentage, pain levels, programintensities, program date, and readiness scores for each of thesubjects. The subjects shown in the user interface 900 a may subjectswho are currently being evaluated. The interface 900 a may be generatedby the computer system 110. The interface 900 a may be populated withdata acquired from the computer system 110. The interface 900 a may beprovided for display by a device associated with a third party 206(e.g., a coach, caregiver, employer, potential employer, supervisor,doctor, researcher, and the like) or other user.

The interface 900 a includes a display area 902 that includes a subjectsection 904 a, a first filter section 906, and a second filter section908. The subject section 904 a includes a list of all subjects andcorresponding analytical data, such as program completion percentage,pain levels, program intensities, program date, and readiness scores foreach of the subjects. The interface provides functionality so a user mayuse a search box in the subject section 904 a to search through thesubjects by name or ID. The interface provides functionality so a usermay use the first filter section 906 to filter subjects presented in thesubject section 904 a by study levels. The interface providesfunctionality so a user may use the second filter section 908 to filtersubjects by their activity/inactivity/adherence with respect to theprogram. For example, inactivity may be defined by a subject failing tocomplete any goals or challenges for a given amount of time, e.g., twodays.

The user interface 900 a also presents an expanded subject section 904b. The expanded subject section 904 b may provide the same informationas the subject section 904 a. The expanded subject section 904 b islarger than the subject section 904 a, which allows the third-party 206or another user to process the information presented in the expandedsubject section 904 b more efficiently than information presented in thesubject section 904 a.

The user interface 900 a also presents additional information such as aprogram day and location (“Site”). The completion percentage for each ofthe subjects may indicate (i) how far along the subject is in theprogram, and/or (ii) if the subject is complying. A low percentage mayindicate that the subject is failing to comply with the program whencompared with the number of days that the subject has bene in theprogram, which could result in a lower readiness score. The level ofpain and intensity may also affect how a readiness score for a subjectis calculated. For example, the level of pain and intensity may indicatethat the subject recovers slowly, recovers quickly, suffers from chronicpain, or the like. These determinations may, in turn, affect a readinessscore for the subject, an expected time until readiness, or the like.

FIG. 9B depicts a user interface 900 b showing assessment analyticaldata for a particular subject 910. This analytical data includes a timeseries that shows the program trends for the subject 910. The interface900 b may be generated by the computer system 110. The interface 900 bmay be populated with data acquired from the computer system 110. Theinterface 900 b may be provided for display by a device associated withthe subject 910, a third party 206 (e.g., a coach, caregiver, employer,potential employer, supervisor, doctor, researcher, and the like)associated with the subject 910, or another user.

The interface 900 b includes a subject information section 912. Thesubject information section 912 includes information corresponding tothe subject 910, such as an ID, a gender, contact information, a programenrollment date, and a type of training plan.

The interface 900 b includes timeline sections 914 a and 914 b. Thetimeline sections 914 a and 914 b each present a time series for thesubject 910. The time series for the subject 910 indicates the activityof the subject 910 over the course of the program/assessment. Forexample, the time series may show the number of days that the subject910 completed exercises, the number of days that the subject 910 loggedon but failed to complete their exercises, and the number of days thatthe subject 910 was inactive over the course of the program. Thisinformation may be used by the computer system 110 to determine, forexample, that the subject 910 has completed a high percentage of theirexercises, has completed a low percentage of their exercises, has animproving exercise completion percentage, has a worsening exercisecompletion percentage, or the like. The computer system 110 may then usethese determinations in generating predictions such as a readiness scorefor the subject 910, a time until the subject is ready, or the like. Theinterface 900 b may be presented to a coach, caregiver, or another useras a virtual check-in of the subject 910.

The interface 900 b may be presented to a user when the user selects aparticular subject from the list of subjects in the interface 900 ashown in FIG. 9A. The interface 900 b may be presented to thethird-party 206 when the third-party 206 selects a particular subjectfrom the list of subjects in the interface 900 a shown in FIG. 9A.

FIG. 9C depicts a user interface 900 c of subject activity and riskmanagement. The user interface 900 c may allow for the display andcomparison various subjects and their respective ratings against risk,such as incomplete trainings, pain, and aversion. As shown, the userinterface 900 c includes a number of graphical representations of thesubjects' progress in the program. These graphical representations show,for example, the trends of the subjects' activity, workout or challengecompletion percentages, pain levels, and feelings. The interface 900 cmay be generated by the computer system 110. The interface 900 c may bepopulated with data acquired from the computer system 110. The interface900 c may be provided for display by a device associated with a thirdparty 206 (e.g., a coach, caregiver, employer, potential employer,supervisor, doctor, researcher, and the like) or another user associatedwith the group of subjects.

The interface 900 c includes a number of display areas: a first displayarea 920 that presents user activity and/or risk management data, asecond display area 930 that presents site overview and/or projectmanagement data, and third display area 940 that presents workoutratings and/or insight.

The display area 920 includes a current enrollment section 922 showingthe current number of subjects enrolled in the program and the targetnumber of subjects. Here, the number of subjects enrolled is thirty andthe target number of subjects is 196. The display area 820 also includessecond section 924 that provides additional program information such asthe number of additional subjects needed to meet subject target, thenumber of subjects currently enrolled, the number of subjects who havesuccessfully completed the program, and the number of subjects inmaintenance, e.g., the number of subjects continuing to use the programapplication after completing the program.

The display area 930 includes an enrolled user activity section 932showing a graph depicting the activity of subjects. The graph maydisplay an indication of the number of subjects who are adherent, whoare non-adherent, and/or who have completed the program over a giventime period. For example, the graph shows the number of subjects thatwere adherent, e.g., completed their challenges, and non-adherent, e.g.,failed to complete their challenges, for each day between 01/10/19 and01/17/19. The display area 930 also includes a first summary section 934that indicates the number of subjects that are active and inactive. Thedisplay area 930 further includes a second summary section 936 thatindicates the number of subjects that are adherent and non-adherent.

The display area 940 includes a training section 942 showing a graphthat indicates the percentage of workouts that the subjects completedover a given period of time. The display area 940 also includes a painlevel section 944 showing a graph that indicates the pain level'sreported by the subjects over a period of time. The display area 940further includes a feeling section 946 showing a graph that indicateshow the feelings reported by the subjects over a period of time.

The data and trends depicted in the user interface 900 c may be used bythe system 100 (e.g., by the prediction and scoring module 114 of thecomputer system 110) in calculating readiness scores for the subjects oran average score for the group or team of subjects, performance trendsfor the subjects or an overall performance trend for the group or teamof subjects, a confidence that the subjects will likely achievereadiness or a confidence if the group or team of subjects will likelyachieve readiness, a time when each of the subjects will likely achievereadiness or a time when the group or team of subjects will likelyachieve readiness, or the like.

The information displayed in the user interface 900 c can be collectedfrom the self-reported information 150, from the sensor data 106, and/orfrom the third-party input 152 shown in FIG. 1C. In someimplementations, the information displayed in the user interface 900 cis only collected from the self-reported information 150.

The information displayed in the user interface 900 c can be generatedby the data aggregation module 112 shown in FIGS. 1A-1C. The dataaggregation module 112 may, for example, organize the self-reportedinformation 150, the sensor data 106, the third-party input, and/orother data by the date (e.g., day as shown), by the group of subjects,by the team of subjects, and/or by a particular location of subjects.

The information displayed in the user interface 900 c may be provided tothe scoring and prediction module 114 shown in FIGS. 1A-1C. All or partof the information displayed in the user interface 900 c may be part ofthe subject data 116.

FIG. 9D depicts a user interface 900 d of challenges or goals, andcorresponding analytics data. The interface 900 d may be generated bythe computer system 110. The interface 900 d may be populated with dataacquired from the computer system 110. The interface 900 d may beprovided for display by a device associated with a third party 206(e.g., a coach, caregiver, employer, potential employer, supervisor,doctor, researcher, and the like) or another user associated with thegroup of subjects whose data is reflected in the interface.

The user interface 900 d includes a first display area 950 that providesthe most popular or highest rated challenges by subjects. The userinterface 900 d also includes a second display area 952 that providesthe least popular or lowest rated challenges. The user interface 900 dfurther includes a third display area 954 that provides indications ofchallenge completion rates. In the display area 954, a user can selectwhether to view challenges that were started but not yet completed,challenges that were completed, or challenges that have yet to beopened. The display area 954 may also provide an indication of howfrequent the selected activity is with the various challenges, and howthe frequency of the activity of the various challenges compare to oneanother. For example, the display area 954 provides that, in the “Train”category, the item “Daily Exercises” have been started one hundred andseventy-seven times and is, therefore, one or more frequently startedchallenges as indicated by the graph 956.

The user interface 900 d may display various analytics of interventionchallenges or goals. The analytics may indicate what challenges subjectslike, what challenges subjects dislike, what challenges subjects arelikely to perform, and/or what challenges subjects are unlikely toperform. The analytics may be based on feedback from subjects, e.g., abinary indicator such as a thumbs up or thumb down. The analytics mayalso be based on subjects' activity, such as an indication that manysubjects start but don't complete a particular challenge, never open orstart a particular challenge, always start a particular challenge,always complete a particular challenge if started, or the like. Forexample, the positive popularity feedback in the display area 950, thenegative popularity feedback in the display area 952, and the challengecompletion rates in the display area 954 may be provided to the scoringand prediction module 114 shown in FIGS. 1A-1B to suggest challenges fora given subject, to add challenges for a given subject, to removechallenges for a given subject, to remove a challenge for all subjects(e.g., the lowest rated or least popular challenge(s)), to add achallenge for all subjects (e.g., the highest rated or most popularchallenge(s)), or the like. In making these predications, the scoringand prediction module 114 may use one or more static algorithms, or mayuse one or more machine learning models such as the machine learningmodels 130, 132, and 134 shown in FIG. 1C.

The computer system 110 may use the challenge completion percentages orrates from multiple subjects in generating a training set to train oneor more machine learning models. For example, the computer system 110shown in FIGS. 1A-1C may use the challenge completion percentages orrates shown in the user interface 900 d in generating training sets forthe machine learning models 130, 132, and 134, and/or the one or moreintervention model(s) 136 shown in FIG. 1C.

FIG. 9E depicts a user interface 900 e of correlation trends amongmultiple subjects. The user interface 900 e may display correlatingoutcomes and vulnerabilities across all subjects, a group of subjects,or a team of subjects in a display area 960 as a graph 962. The userinterface 900 e correlates the subjects' workout completion percentagewith their pain levels and with their feelings (e.g., good, bad, or noresponse). This information may be used by the computer system 110 totrain one or more machine learning models. This information may indicatetrends that can be used by the computer system 110 to predict, forexample, how a given subject will feel or the pain levels they willexperience if they complete 75% of their assigned workouts. Theinterface 900 e may be generated by the computer system 110. Theinterface 900 e may be populated with data acquired from the computersystem 110. The interface 900 e may be provided for display by a deviceassociated with a third party 206 (e.g., a coach, caregiver, employer,potential employer, supervisor, doctor, researcher, and the like) oranother user associated with the group of subjects.

The interface provides functionality so a user may filter the datapresented in the graph 962 using the filter controls 964. A user maychoose to only see the trends for a given date range by selecting a daterange from the dropdown menu 966.

The user interface 900 e may be generated based on past data of multiplesubjects or the current data of a group of subjects. The correlationtrends depicted in the user interface 900 e may be used by, for example,a static algorithm or by the machine learning models 130, 132, and/or134 shown in FIG. 1C in determining a performance trend of acorresponding subject or a risk of a corresponding subject. For example,the correlation trends depicted may indicate that subjects who havecompleted a lower percentage of workouts should receive a higher riskscore since it indicates that those who complete a lower percentage ofworkouts are more likely to experience higher pain levels which, inturn, causes them to feel bad. Other correlations would likely indicatethat those subjects who experience higher pain levels and/or generallyfeel bad are less likely to complete the program, less likely to improvetheir performance, less likely to reach their goals, etc.

While the example of FIG. 9E shows a specific example of relatingworkout completion percentage, pain levels, and participant subjectiveresponses, the same type of chart can be used to show correlationsbetween any of various parameters or factors measured by the system.This can also be used to show correlations or relative paths ofdifferent activities or subject characteristics to different readinessscores or readiness timing (e.g., ready within 1 month, ready within 2months, etc.).

FIG. 10 is a diagram that illustrates example interfaces 1000 a-1000 b.The interfaces 1000 a-1000 b may be presented to the subject 202 througha client application (e.g., a mobile application) on the client device204 during an assessment of the subject 202. The interfaces 1000 a-1000b may be generated by the computer system 110. The interfaces 1000a-1000 b may be populated with data acquired from the computer system110.

In some implementations, the interfaces 1000 a-1000 b, or interfacessubstantially similar to interfaces 1000 a-1000 b, are presented to thethird-party 206 (e.g., a coach, caregiver, employer, potential employer,supervisor, doctor, researcher or the like) through an application onthe third-party device 208. These interfaces may be generated by thecomputer system 110.

The interface 1000 a includes four display areas: an assessmentduration/date area 1002, a current weather area 1004, a training summaryarea 1006, and a detailed training area 1008. The interface 1000 a alsoincludes an interface element 1010. The assessment duration/date area1002 includes an indication of the subject 202's current date ofassessment, e.g., an indication of how long the subject 202 has beenassessed for. Here, the subject 202 has just completed their fifth weekof assessment. The current weather area 1004 may provide an indicationof the current weather, such as an image that corresponds with theweather (e.g., clouds and sun if partially cloudy, sun if sunny, cloudsif overcast, clouds and rain if raining, etc.) and/or text thatcorresponds to the current temperature. The computer system 110 may pullweather data from an external server and use this in generating thecurrent weather area 1004.

The interface 1000 a displays a tailored training program for thesubject 202 in the display areas 1006 and 1008. The training summaryarea 1006 includes a summary of the subject 202's tailored trainingprogram for the current assessment date provided in the area 1002. Asshown, the summary of the subject 202's tailored training programprovides an indication of the number of drills (e.g., challenges orgoals) that the subject 202 is expected to complete on this day, thetotal number of exercises in those drills, and an estimated time for thesubject 202 to complete all of the drills/exercises. The drills andtheir exercises may be selected for the subject 202 by the computersystem 110. The computer system 110 may calculate an estimated time forthe subject 202 to complete the drills/exercises based on the subject202's past performances and/or based on the performances of others.

The detailed training area 1008 provides a more detailed view of thesubject 202's tailored training program for the current assessment dateprovided in the area 1002. The detailed training area 1008 may provideinformation about each of the drills, such as a name of the drill, thenumber of exercises in the drill, an expected time for the subject 202to complete the drill, a recommend order for the subject 202 to performthe drills in, and whether the subject 202 has completed a given drill.For example, a drill may be marked as completed by an indicator next tothe drill changing color (from white to black). A subject 202 may bepresented more detailed information about a given drill by selecting thedrill. When the subject 202 selects an individual drill, they may bepresented more detailed information on each of the exercises within thedrill, such as a name of the exercise and an expected time for thesubject 202 to complete the exercise.

As shown in the detailed training area 1008, five of the six drills(e.g., challenges or goals) of the subject 202's tailored trainingprogram are provided. These challenges or goals may be for a particularperiod of time, e.g., for the specific assessment date (e.g., day 2 ofweek 5), for the current week, for the current month, etc. The subject202 may select a challenge or goal (e.g., through a touch input) to seeadditional details for the selected challenge or goal.

The subject 202 may select interface element 1010 (e.g., through a touchinput) which can initiate the start of the first challenge or goal,e.g., the “Preparation Drill” consisting of ten exercises, or the startof the next incomplete challenge or goal, e.g., the “Military MovementDrill” consisting of three exercises.

The interface 1000 b allows for the subject 202 to enter and submitself-reported responses. The interface 1000 b includes a display area1012 where the subject 202 can provide responses for indicating a painlevel experienced by the subject 202 after completing a set ofchallenges or goals (or completing a given challenge or goal), e.g., thesix drills for week five day 2 of the subject 202 assessment. Thedisplay area 1012 includes a slider 1014 that the subject 202 canoperate by sliding to the left to indicate less pain experience or bysliding to the right to indicate greater pain experienced. The operationof the slider 1014 is one type of response that the subject 202 canprovide. The responses may include a binary indication of how thesubject 202 feels, e.g., good or bad. As shown, the interface 1000 balso includes interface elements 1016 and 1018. The interface element1016 allows the subject 202 to provide a binary response that they feelbad. The interface element 1018 allows the subject 202 to provide abinary response that they feel good. The subject 202 may be able toprovide responses both through the slider 1014 and either the interfaceelement 1016 or the interface element 1018. The application may promptthe subject 202 with the interface 1000 b after the subject 202 attemptsor successfully completes a challenge or goal, after the subject 202 hasattempted or successfully completed all challenges or goals for a givenassessment date, or after an assessment date has passed (e.g., responsesmay be requested from the subject 202 the next day). The responses maybe provided from the client device 204 to the computer system 110. Thecomputer system 110 may use the responses to modify the tailoredtraining program for the subject 202.

FIG. 11 is a diagram that illustrates example interfaces 1100 a-1100 b.The interfaces 1100 a-1100 b may be presented to the subject 202 througha client application (e.g., a mobile application) on the client device204 during an assessment of the subject 202. The interfaces 1100 a-1100b may be generated by the computer system 110. The interfaces 1100a-1100 b may be populated with data acquired from the computer system110. The assessment may occur based on a set schedule, e.g. may occur ata time set by the third-party 206 (e.g., a coach, caregiver, employer,potential employer, supervisor, or the like), may occur every week(e.g., on the last day of the week), may occur every month (e.g., on thelast day of the month), etc.

In some implementations, the interfaces 1100 a-1100 b, or interfacessubstantially similar to interfaces 1100 a-1100 b, are presented to thethird-party 206 (e.g., a coach, caregiver, employer, potential employer,supervisor, doctor, researcher, or the like) through an application onthe third-party device 208. These interfaces may be generated by thecomputer system 110.

The interface 1100 a displays prior assessment data for the subject 202and current metrics for the subject 202. The interface 1100 a includestwo display areas: an assessment duration/date area 1102, and anassessment area 1104. The assessment duration/date area 1102 includes anindication of the subject 202's current date of assessment, e.g., anindication of how long the subject 202 has been assessed. Here, thesubject 202 has just completed their fifth week of assessment. Theassessment area 1104 includes a prior assessment data section 1106, acurrent metrics section 1108, and a section that includes an interfaceelement 1122. The prior assessment data may include, for examples,predictions such as if they are likely to reach a particular readinessscore and, if so, when they are expected to reach that readiness score.The prior assessment data may include prior readiness scores,performance trend(s), prior risk scores, prior exercise completionpercentage, prior average pain levels, prior average feelings, etc.

The prior assessment data section 1106 includes a first predication 1110and a second prediction 1112 that were generated using, for example, theprior assessment data shown in a table 1114. The predictions can begenerated by the computer system 110. As shown, the table 1114 includesdata on the corresponding subject 202's first four weeks of assessment.

The current metrics section 1108 includes multiple metrics 1116-1120corresponding to the subject 202 that were calculated using datacollected over the current assessment period. The metrics 1116-1120 mayhelp to show the subject 202 and/or the third-party 206 areas where thesubject 202 is improving or areas where the subject 202 still needs toimprove, e.g., when compared with the data in the table 1114. Here, themetric 1116 is the exercise completion percentage for the subject 202over the current assessment period. The metric 1118 is the average painlevel for the subject 202 over the current assessment period. The metric1120 is the recent average feeling for the subject 202 over the currentassessment period. The metrics 1116-1120 may be calculated, for example,by the computer system 110 shown in FIGS. 1A-1B. These metrics may becalculated by the computer system 110 based on the self-reportedinformation 150 and/or the sensor data 106.

In the example of FIG. 11, the assessment period is one week. However,other assessment periods are possible. For example, an assessment may beperformed every day, every two weeks, every month, or the like. Inaddition, there may be multiple assessments. For example, there may be adaily assessment that is, for example, provided to the subject 202through the client device 204, and a weekly assessment that is providedto the subject 202 through the client device 204 and to the third-party206 through the third-party device 208.

The computer system 110 shown in FIGS. 1A-1C may use this priorassessment data in determining a new readiness score for the subject202, a new risk score for the subject 202, an updated performance trendfor the subject 202, and/or one or more predictions (e.g., whether thesubject 202 is still likely to reach the readiness score goal, and, ifso, what the updated expected time is for the subject 202 to reach thereadiness score goal). The computer system 110 may be used to generatethe predictions 1110 and 1112. The computer system 110 may be used togenerate the table 1114. The computer system 110 may be used tocalculate the metrics 1116-1120.

The interface 1100 a also includes the interface element 1122. When thesubject 202 or the third-party 206 selects the interface element 1122,assessment results may be generated for the subject 202 and the subject202/third-party 206 may be presented the interface 1100 b. Theassessment results may be those for the current assessment period, e.g.,for week five. The assessment results may be generated by the computersystem 110.

The interface 1100 b displays information to the subject 202 based onthe new assessment. The interface 1100 b displays a current readinessscore for the subject 202, a current risk score for the subject 202, andpredications for the subject 202 in a summary section 1124. For example,as shown, the interface 1100 b displays a prediction that the subject202 is likely to reach the goal of obtaining a readiness score of 8/10earlier than previously expected, at the beginning of week 8 of thesubject 202's program. With respect to FIG. 1A, the readiness score,risk score, and predications may be determined by the scoring andprediction module 114, e.g., based on the outputs of the machinelearning models 130, 132, and/or 134 shown in FIG. 1C. With respect toFIG. 1C, the readiness score, risk score, and predications may bedetermined by the machine learning models 130, 132, and/or 134.

The interface 1100 b also displays multiple options 1126 and 1128 thatthe subject 202 or the third-party 206 may select. A selection of eitherof the options can result in the subject 202's program being changed toreflect the changes indicated in the respective option 1126 or 1128. Insome implementations, the options 1126 and 1128 are not provided to thesubject 202 for selection. For example, the options 1126 and 1128 may bepresented to the third-party 206 such as a coach or caregiver of thesubject 202. The coach or caregiver may then select one of the options1126 or 1128, or may choose not to select an option. With respect toFIG. 1C, the options 1126 and 1128 may be generated by the interventionsmodel(s) 136 as recommendations, e.g. based on a determination that theywould have the effect of getting the subject 202 to their readinessscore goal earlier than currently expected. In addition, theintervention model(s) 136 may calculate an expected effect if either ofthe options 1126 or 1128 are chosen. For example, as shown, theintervention model(s) 136 may have calculated that if option 1126 isselected, the subject 202 is expected to reach their readiness scoregoal five days earlier.

FIG. 12 is a flowchart that illustrates an example process 1200 forpredicting readiness. The process 1200 can be performed by one or morecomputers, such as using the computer system 110 described above. Stepsof the process 1200 can be performed by one or more servers, one or moreclient devices, or by a combination thereof and/or other devices.

The process 1200 includes accessing a database to obtain status datathat indicates activities or attributes of a subject (1202). The statusdata can include data provided by an electronic device to the one ormore computers over a communication network. As discussed above, adiverse set of subjects can be evaluated and assessed using thetechniques herein, such as a device, a system, a model, a hardwarecomponent, a research study, a software component, an organization, ateam, an individual (e.g., a person), a combination of one or morepeople and equipment, and so on.

The attributes indicated in the database can include any of varioustypes of information that describe or characterize the state of thesubject, and the attributes can be derived from many different sources.The attributes can describe, for example, physical characteristics(e.g., size, dimensions, weight, age, maintenance status, health status,physiological measures, genomics data, proteomics data, etc.) orfunctional characteristics (e.g., capacity, speed, efficiency,reliability, etc.). The attributes can be self-reported by a subject,provided by a third party (e.g., an administrator, a coach, atechnician, a doctor, a researcher, a supervisor, etc.), provided by oneor more devices (e.g., devices used in training, tools, networkingequipment, maintenance devices, medical equipment, phones, wearabledevices, devices that interact with the subject or monitor the subject,etc.).

As a few examples, when the subject is a device, attributes can beindicators of the structure and/or functional ability of the device. Forexample, attributes can indicate various measures of status orperformance capability, such as storage capacity, processor usage, powerconsumption, memory usage, network bandwidth usage, network latency,response latency, throughput, error rates, and so on. Attributes canalso refer to specifications or ratings of the device and itscomponents; the make, model, or type of device; the number and types ofcomponents included in the device; hardware and/or softwareconfiguration of the device; configuration settings; and so on.

For a subject that is a person, attributes can include vital signs orbaseline indicators of health status (e.g., heart rate, respiratoryrate, blood pressure, oxygen saturation, respiratory rate, respiratoryeffort, capillary refill time, temperature, etc.). Other attributesinclude height, weight, strength measurements, endurance measurements,blood chemistry measurements, genomics data (e.g., data indicatingwhether the subject has or lacks certain genes or certain classes ofgenes, indications of which form of a gene is present, etc.), proteomicsdata (e.g., data indicating the identification, localization, andfunctional characteristics of proteins of a subject). Subject attributescan include whether a person has been diagnosed with a disease or othercondition (e.g., cancer, diabetes, heart disease, chronic obstructivepulmonary disease (COPD), etc.), the current status of the disease(e.g., disease stage or classification, effects or limitations due tothe condition, a severity or level of progression of the disease),whether the person has received treatment and what type of treatment,and so on. Attributes may indicate the structure and/or functionalcapability of any structures or systems of the body. Subject attributescan include mental and psychological indicators such as anxiety levels,pain levels, scores for various personality traits, and so on.

The database can include data generated for the subject over a period oftime, the status data comprising information about activities orattributes of the subject at multiple points in time. The computersystem 110 track or monitor the activities of subjects over time andcollect information obtained. For example, the computer system 110 maycommunicate with various other devices to track different aspects of anactivity (e.g., type of activity, duration of the activity, intensity ofthe activity, results of the activity, effects of activity on thesubject, etc.) Individual instances of activities may be tracked and/orcomposite measures of activities (e.g., frequency, averagecharacteristics, etc.) can be tracked. Subjects can be monitored todetect changes in attributes as well.

In some implementations, the status data may include sensor datacollected by one or more of the sensing devices 104 a-104 c and sent tothe computer system 110. The sensor data may include data collectedwhile the subject, e.g., person, is engaged in a particular activity, inparticular, activities such as training activities or assessments. Thestatus data may include data self-reported by the subject, datacollected from others, data from sensors, etc.

In some implementations, for example, when the subject is a person, thestatus data may include one or more of heart rate data of the subject,oxygen saturation data of the subject, data indicating an exercisedistance that the subject ran or walked, data indicating an exerciseintensity that was performed by the subject, or data indicating aduration of an exercise that was performed by the subject. As discussedabove, these can be current or recent values (e.g., the most recentlymeasured) and/or prior values (e.g., a series of historical measurementstaken at different times). These various data types and other may becollected using multiple sensors and/or devices, for example the sensingdevices 104 a-104 c.

The activities tracked by the computer system 110 can include actionsthat involve or are performed by the subject. Some tracked activitiesmay be directly related to the capability to be developed in thesubject, e.g., the readiness criteria used to evaluate the capabilitiesof the subject. For example, a program for enhancing the capability ofthe subject may include training, practice, tests, and other actionsdesigned to improve the condition or capability of the subject. Thecontext of the activities (e.g., the location, date, time day, resourcesallowed, whether performed in a group or alone, who supervised orinstructed the activity, etc.) can be tracked and recorded as well. Sometracked activities may not be specifically related to the capability tobe developed or may not initially be known or expected to be related tothe capability.

The computer system 110 can track attributes or activities of each ofmultiple subjects over a period of time, as well as changes in levels ofcapability of the multiple subjects over the period of time. Thecomputer system can the train models based on the tracked data. Bytracking many variables (e.g., subject attributes, subject activities,context of the subject and activities, etc.) for many subjects andstoring the data in the database, the computer system 110 can obtain arich data set with which to discover elements that have relevance to theprocess of a subject acquiring readiness and the manner those elementsimpact readiness. This data, whether used for machine learning trainingor through direct analysis and extraction of relationships by thecomputer system 110, can be used to identify which features arepredictive of different types of readiness (e.g., different capabilitiesof subjects) and to generate models that can be used to make predictionsbased on those features.

Some tracked data may represent factors that do not seem related to asubject's readiness or capabilities, but nevertheless may be discoveredby the computer system to affect the process of at least some subjectsin gaining readiness in at least some contexts or circumstances. Thecomputer system 110 can use the examples in the tracked data to identifythe relative impact of tracked data items on different capabilitiesbeing acquired, how they affect the instantaneous capability vs. thetrend or trajectory of acquiring readiness over time, how tracked itemsaffect readiness in combination, how tracked items enhance or diminishthe effects of activities in gaining readiness, how tracked data itemsvary in importance for subjects having different combinations ofattributes, and so on.

For example, in the case of athletes, factors such as amount of sleepand diet may affect subjects that have certain genes, fitness levels,experience levels, training histories, behaviors, or other attributesmore than other subjects that do not have the same attributes orcombinations of attributes. Similarly, tracked factors may affect theacquisition of some capabilities more than others (e.g., sprinting speedvs. distance running speed vs. weight lifting ability), and may havediffering impact to enhance, diminish, or have no effect on differenttraining activities.

The process 1200 includes deriving a set of feature scores from thestatus data for the subject (1204). The feature scores may indicateattributes or activities of the subject. The attributes can be currentattributes or former attributes, and so can reflect the progress orchange experienced by the subject over time. Similarly, the activitiesindicated can be current or former activities, such as a set ofactivities performed to enhance the subject's capability (e.g., trainingactions, tasks attempted or performed and associated outcomes, etc.).Some feature scores may be based on user interaction with a device, suchas user input to a graphical user interface, responses to questions,entered text. Other feature scores may be based on data collected inother way

The feature scores can be based on sensor data that is acquired by oneor more sensors during one or more activities of the subject or thatindicates one or more attributes of the subject. For example, the sensordata can indicate measurements or detection of attributes and activitiesof the subject, and the feature scores can be the measured or detectedvalues or other values derived from them (e.g., sensor measurements thathave been normalized, quantized, rounded, combined, or otherwiseadjusted, or classifications based on the measurements). Examples ofsensors that can provide data include accelerometers, proximity sensors,temperature sensors, pressure sensors, optical sensors, cameras, GPSreceivers, and so on. Examples of sensing devices that can measurephysiological parameters include oximeters, glucometers,electrocardiogram sensors, heart rate sensors, electroencephalogramsensors, electromyogram sensors, and respiration rate sensors.

The set of feature scores can include a value for each of multiplepredetermined feature types. The feature types can be a predeterminedset of data types that have been previously determined by the computersystem, for example, as part of analyzing data in the database andconstructing the one or more models. The computer system can store dataindicating the set of features types (e.g., data types or types ofmeasures to be generated) that correspond to each model. For example,different models for different types of subjects may use feature scoresrepresenting different types of information about subjects.

In the example of FIG. 1C, the feature scores may be derived by thecomputer system 110, e.g., using the data aggregation module 112 toextract and/or organize the activity data. The feature scores mayinclude all or part of the aggregated data 122 shown in FIG. 1C,converted to an input format that the model is configured to accept(e.g., binary values, integers, or other values in a predeterminedrepresentation). The feature scores may indicate attributes of theperson such as, for example, physiological attributes, such as height,weight, blood pressure, health status, and psychological attributes.

To derive the appropriate set of feature scores, the computer system 110can select one or more models that are appropriate for the subject,readiness criteria for the subject, and the type of prediction to bemade. For each model, stored data can indicate the feature types to beused with the model, such as the types of information to be provided andthe format for providing the data to the model (e.g., binary values,integers, floating point values, an appropriate level of precision,etc.). Once the computer system 110 selects a model to use, the computersystem 110 can access the data specifying the input feature types forthat model and generate them in the format the model is designed toaccept.

In general, as part of generating predictive models, the computer system110 can assess the predictive ability of different data types toindicate readiness, potential for future readiness, and other factors.From this analysis, the computer system can select a subset of theavailable types of data about subjects to use as features providinginputs to the predictive models. The subset can be a proper subset,e.g., fewer than all of the available types of data. The analysis can beperformed at a fine-grained level, for example, with a different set offeature types used for each different type of subject, for eachdifferent capability or aspect of readiness to be predicted, and/or foreach type of prediction (e.g., current readiness, future readiness,maximum potential readiness, time to reach a particular readiness level,etc.). Indeed, the computer system 110 can train different models fordifferent subjects, readiness criteria, and/or types of predictions,with each model having a corresponding set of feature types representinga subset of the types of information available in the database.

The varied information in the database can also be used to selectinterventions to enhance readiness. Even if multiple options (e.g.,different training programs) tend to increase the capabilities ofsubjects, different options will nevertheless vary in theireffectiveness for individual subjects, which have different combinationsof attributes and experience. An activity that provides appropriateimprovement for one subject may result in too slow of improvement foranother subject, or may have a lower likelihood of being carried out byone subject than another. Based on the information in the database, thecomputer system 110 can generate models, tables, equations, or otherelements that can be used to predict the effectiveness of differenttraining activities or other factors (e.g., different coaches, differentenvironments, different resource levels available to subjects, etc.)given a subject profile (e.g., a combination of attributes andactivities). This can be done with machine learning models that trainedto output, given input feature scores indicating subject attributesand/or activities, scores indicating the appropriateness of a trainingactivity or other intervention (e.g., a communication to the subject, anoutput by or a command to a device associated with the subject, a changein behavior of the subject, a notification for a trainer or technicianassociated with the subject, etc.). This can also be done by generatingdata for a table indicating the effects of attributes, activities, orother data on readiness, such as shown in the table 400 of FIG. 4.

The set of feature scores derived from the status data can includeinformation about different attributes and/or activities of the subjectover time. For example, the feature scores may indicate the activitiesor attributes of the subject at each of multiple points in time. Forexample, different instances of activities (e.g., training activities,attempts to perform a task, etc.) can each be provided so they can beused in making predictions. Similarly, the values or measures indicativeof different subject attributes at different times can also be provided.In some cases, information about different points in time, e.g., timeseries data for a subject, can be provided by concatenating multiplesets of feature values, each representing attributes and activities atdifferent times, into a single input feature vector or input data setfor a model. As another example, multiple sets of feature values can beprovide to a model separately, e.g., sequentially as a sequence ofmultiple feature vectors, and the model can include a memory capabilityto retain information from the processing of one input vector to thenext. A recurrent neural network, such as one including long short-termmemory (LSTM) blocks, or other model can be used for this purpose.

The process 1200 includes providing the feature scores to one or moremodels (1206). For the example of FIGS. 1A-1C, the feature scores may beprovided to the scoring and prediction module 114 that includes themachine learning models 130, 132, and 134. The one or more models can bemodels configured to predict readiness of subjects to satisfy one ormore readiness criteria.

The one or more readiness criteria can represent a predetermined stateor level of capability that can be used as a reference to evaluate thesubject. For example, the readiness criteria can define a desiredphysical state of the subject (e.g., certain physical attributes), adesired functional state of the subject, and/or a desired level ofcapability of the subject to perform a task. The readiness criteria maybe defined in any of various different ways. For example, the readinesscriteria can specify a target state in which a subject is capable ofperforming a task or action. As another example, the readiness criteriacan specify that a subject should be capable of performing a task oraction in a particular manner (e.g., satisfying a predeterminedthreshold for speed, latency, accuracy, precision, consistency,efficiency, reliability, scalability, capacity, throughput, volume,intensity, etc.) or according to a performance standard. The readinesscriteria may include multiple components, for example, they may requirea subject to have the capability to perform each of multiple differenttasks to reach the standard of readiness specified by the readinesscriteria.

The one or more models may include one or more machine learning models.The one or more models comprise at least one of a neural network, asupport vector machine, a classifier, a regression model, a clusteringmodel, a decision tree, a random forest model, a genetic algorithm, aBayesian model, or a Gaussian mixture model. The one or more models mayinclude statistical models, rule-based models, and other types ofmodels. The feature scores may be provided as a feature vector, e.g., asequence of values where each value corresponds to an input slot (e.g.,a portion of an input layer of a neural network) designated for acertain type of data.

In some implementations, the one or more models have been trained basedon training data indicating (i) activities or attributes of othersubjects and (ii) outcomes for the other subjects with respect to thereadiness criteria. The training can use the respective progressions ofreadiness of the other subjects over time to configure the one or moremodels to predict readiness of subjects to satisfy one or more readinesscriteria.

The one or more models can be trained, based on data from the databaseabout multiple subjects, to predict the readiness of the subject at thecurrent time or at a time in the future. The models can be configured tomake these predictions using current information about the subject,historical information about the subject, and/or information aboutplanned or anticipated future activities of characteristics of thesubject (e.g., a training plan, estimated improvements in a capabilitybased on current trends or patterns, etc.) In general, the data in thedatabase can indicate, for each of multiple subjects, a variety of datapoints indicating attributes of the subjects, activities of the subject,and other information.

The data in the database can provide examples of the patterns, trends,and/or progressions in achieving readiness that subjects experienced.For example, the data can indicate, for subjects of differentcombinations of attributes, which activities enhanced readiness (e.g.,performance capabilities) as well as to what extent they enhancedreadiness. This can indicate, for example, how soon different activitiesor actions improved capabilities, which combinations were most effective(e.g., which combinations provide synergies), which subject attributesmade subjects more or less likely to improve with different activities,the magnitude of the effect that different activities and attributeshave on readiness, and so on. The computer system can analyze theexamples in the database to determine these relationships, eitherthrough explicit analysis or through machine learning training, e.g., sothat a model implicitly learns the predictive value of different dataitems on current or future readiness. Training can incrementally oriteratively update the values of parameters in the models to learn theimpact of different factors on predicted outputs. In the case of neuralnetworks, backpropagation can be used to alter neural network weightsfor neurons at various layers of the neural network model.

Models may be trained to predict future readiness by, among othertechniques, training with input-target pairs with a mismatch between theinputs and targets. For example, the data indicating attributes, pastactivities, and/or planned future activities for subjects after onemonth of a training process can be used to generate input features,while data indicating the measured, actual readiness of those subjectsafter three months of the training process can be used at the outputtargets. By training with these data pairs, the computer system 110 cantrain a model to use the information about subject at one time period tooutput the results achieved at a different time period. Of course,similar techniques can be used without a mismatch between input andtraining targets to train a model to predict the readiness at thecurrent time, e.g., so that models learn to predict current readinessbased on current input data.

Another technique that can be used to train the models is to use thelongitudinal progress of subjects to learn the patterns, trends, andprogressions of readiness over time. For an individual subject with datacollected at 10 different weekly intervals, this can provide tendifferent examples for training a model to predict readiness at the10-week time. Each example can reflect cumulative interactions of priorweeks. For example, the data at the end of the first week (along with avalue indicating that the input corresponds to the first week) can beused as on training input, with the eventual 10-week readiness score asthe training target. The data at the end of the second week (along witha value showing that the input corresponds to the second week) can beused as on training input, with the same eventual 10-week readinessscore as the training target. Data from other weeks can be used in thesame manner to create other training example data points. By training inthis manner, using the examples of many different subjects havingdifferent attributes and experiences, the model can learn the typicaltrends of readiness progression so it can predict readiness based ondata representing different time periods or stages in acquiringreadiness. The training process can also indicate the variability inthese patterns and progressions, allowing the model to output, forexample, a confidence level for predictions.

The process 1200 includes providing, based on output of the one or moremachine learning models, output indicating the subject's readiness(1208). With respect to FIGS. 1A-1C, the output indicating the subject'sreadiness may be outputted by the scoring and prediction module 114. Theoutput may be a readiness score. For example, in some cases, the outputis a readiness score indicating a predicted physical or informationprocessing performance of the subject, whether representing the currenttime (e.g., the time corresponding to the most recent feature values forthe subject) or a future time. In some cases, the output is a readinessscore indicating a combination of both the physical and informationprocessing performance of the subject. The output may take into accountthe subject's current physical and/or mental performance as well as thesubject's performance trend (e.g., history of progression in measuredreadiness or predicted readiness score for multiple previous times).

Models can be generated to make different types of predictions (e.g.,completion readiness time (CRT), future readiness probability (FRP),altered course success rate (ACS), etc.) with respect to differentreadiness criteria. Typically, a different model can be generated foreach type of prediction to be made. Models can be generated for eachtype of readiness criteria also. A first model can be generated to makeFRP predictions for one set of readiness criteria, and a second modelcan be generated to make FRP predictions for a different set ofreadiness criteria. In some implementations, a model may be configuredto predict readiness with respect to any of multiple different readinesscriteria, with the readiness criteria being specified as an input at thetime of generating a prediction. For example, a model may be trained topredict readiness for physical capability of a person, and variousparameters indicating the level of fitness (e.g., an amount of weight tobe lifted, a time for running a certain distance, etc.) can be providedwith the feature data as input to the model, allowing the model totailor each prediction for the readiness criteria specified. The modelcan be so configured by training the model with examples that involvedifferent readiness criteria and by providing values indicating theapplicable readiness criteria during training.

The output may be a predicted time of when the subject will achievereadiness to satisfy the one or more readiness criteria. The predictioncan take into account various factors which can be reflected in thefeature values, e.g., current attributes of the subject, priorattributes of the subject, a history of activities of the subject, aplanned or scheduled activities of the subject, resources available toor assigned to the subject (e.g., equipment, hardware, software,technicians, doctors, consumables, training courses, time, financialresources, etc.).

The output may be a likelihood or prediction whether the subject willachieve readiness to satisfy the one or more readiness criteria. Thiscan be provided in the form of a binary prediction (e.g., yes or no), aconfidence score (e.g., a value indicating an 80% confidence in aclassification), a likelihood (e.g., a value indicating a 70% likelihoodof achieving readiness), and so on. The prediction may be whether thesubject will ever achieve readiness (e.g., has the capacity to everacquire the reference level of ability represented by the readinesscriteria) or whether the subject will achieve readiness in a certainscenario or given certain conditions. The model can take into accountone or more constraints, whether built into the model through trainingor specified as in input to the model. For example, the constraints maylimit at least one of, a time to achieve readiness, resources availableto the subject, or a training plan (e.g., planned, scheduled, oranticipated training activities) for the subject.

As an example, the constraints used in a generating a prediction mayspecify a future time that readiness potential is being judged (e.g., aspecific date, or an amount of time in the future, such as one month).The constraints may also condition the prediction on factors such as theavailability of certain resources or limitation on the use of certainresources, the elements planned for assisting the subject in achievingcapability (e.g., assuming that a certain set or type of trainingactivities will take place), and so on. Thus, the system enables a userto obtain a prediction whether a subject will satisfy the readinesscriteria at a defined time in the future, or if the system will ever beable to satisfy the readiness criteria.

In general, any of the predictions discussed in this document may beoptionally generated based on one or more constraints. In some cases,these constraints may be designed into a model at the time of training.For example, a model may be trained to predict whether readiness will beachieved within one month. In some cases, the constraints may beindicated to the model, e.g., as input parameters along with featurescores for a subject. For example, a model may be trained to providepredictions based on variable time frames, with the time frame ofinterest being specified as input to the model at the time of inferenceprocessing rather than training. In some cases, constraints may beaccounted for through post-processing of model outputs, combiningoutputs of multiple models, or through other techniques.

Other types of predictions can be generated, based on feature scoresderived from the database, with appropriately trained models. Forexample, models can be configured to predict the type and amount ofresources or training activities that would be needed for individualsubjects to reach the readiness criteria.

The process includes providing output data based on the prediction(1210). The output can be configured to, for example, alter a userinterface of a device or alter interaction of a device with the subject.The user interface may be a user interface of, for example, a phone orcomputer used by the subject or a person associated with the subject(e.g., a coach, a doctor, etc.).

Interaction of a device with a subject can include changing how a devicemonitors a subject, e.g., change the type, frequency, precision, ormanner of obtaining measurements about a subject. For example, if aprediction indicates that at subject is not meeting a readiness target,the frequency of monitoring the subject may be increased and additionaltypes of data may be collected. As another example, for a subject thatis a person, the manner in which a phone, wearable device, or otherdevice interacts with the subject can be changed. For example, any ofvarious interactions can be initiated, discontinued, or modified, e.g.,audible outputs, visible outputs, haptic outputs, reminders,recommendations, surveys, practice or training activities administeredusing the device, etc. In some implementations, the actions of medicalmonitoring or treatment devices can be changed, e.g., to adjustfrequency or dose of medication or to adjust the process of measuringphysiological parameters.

In some implementations, an indication (e.g., a score, color, image,icon, graph, text, etc.) of the prediction is provided to the subject,to a device associated with the subject (e.g., a tool, phone, wearabledevice, monitoring device, etc.), to a person associated with thesubject (e.g., a technician, administrator, coach, doctor, researcher,etc.), or to a device associated with the subject. The indication of theprediction can be provided in a visualization, such as a chart, graph,timeline, scatterplot, map, table, etc. that shows past, present, and/orfuture readiness of the subject. The indication can be provided throughvarious channels, such as an e-mail, a text message (e.g., short messageservice (SMS) or media message service (MMS)), a notification, an alert,a web page, and/or content of an application (e.g., a mobile deviceapplication or computer application). The output data can be provided toa local device or to a remote device, such as over a computer networksuch as the Internet.

As an example, a score indicative of the prediction can be provided fordisplay on a user interface. The output data can include an indicator ofthe prediction for display on a user interface. The output data caninclude visualization data, generated using the prediction, forillustrating a timeline or trend of predicted readiness of the subjectover time. The output data can include a recommendation, determinedbased on the prediction, of an action to improve or accelerateacquisition of readiness of the subject to satisfy the one or morereadiness criteria. For example, the recommendation can include aproposed change to a plan for enabling the subject to reach readiness(e.g., an upgrade plan for a network, a maintenance plan for a device, atraining plan, a medical treatment plan, etc.). The output data caninclude data to cause display of one or more interactive user interfacecontrols that, when selected by a user, alter one or more plannedactions (e.g., a training plan) for assisting the subject to achievereadiness to satisfy the one or more readiness criteria.

The output data can include data to cause display of one or moreinteractive user interface controls that enable a user to select fromamong multiple options for assisting the subject to achieve readiness tosatisfy the one or more readiness criteria (e.g., different activities,resource allocations, configuration changes, medical procedures, etc.).The output data can include an indication of a predicted change inreadiness of the subject, relative to a level of readiness indicated bythe prediction, for each of one or more actions with respect to thesubject. In other words, different options can be provided, each with apredicted effect of selection of that option, e.g., an indication of howmuch more quickly or more efficiently the subject would achievereadiness, or an indication of a predicted change to the subject'sreadiness score if the option is selected. For example, an option mighthave a corresponding indicator that the time to reaching readiness wouldbe reduced by an amount (e.g., 10%, 20%, etc.) or a certain amount oftime (e.g., one week, three days, etc.). To obtain these predictions, analtered set of features scores can be determined for each option, withthe candidate action or activity being reflected in the feature scores.The trained models discussed above can then be used to determine thepredictions of the ultimate effect of altered training plans thatinclude different selectable options. The results of these can becompared to the predictions made without those training options, to showthe relative amount of change (e.g., improvement) that each option hasthe potential to provide.

In some implementations, the computer system 110 computes a readinessscore, for example, a numerical indication such as a number within arange (e.g., a number between and including 0-10). There may be multiplereadiness scores generated for a subject to represent predictedreadiness for different criteria and/or to represent readiness atdifferent times. For example, there may be a readiness score for thesubject's physiological performance, e.g., that indicates how physicallyfit or in shape the subject is. There may be a readiness score for thesubject's psychological performance, e.g., that indicates how mentallystable, resilient, or prepared the subject is, either generally or for aspecific task. There may also be an overall readiness score that iscalculated using readiness scores for the subject's physiological andpsychological performance. The overall readiness score may be acombination of the readiness scores for the subject's physiological andpsychological performance. The overall readiness score may be calculatedby applying a first weight to the physiological readiness score and asecond, different weight to the psychological readiness score, andcombining the weighted scores. For example, for a particular job role,the physiological readiness score may be weighted 70% and thepsychological readiness score may be weighted 30%.

In some implementations, in determining a readiness score, one or morepast performances of the subject may be analyzed. These pastperformances may indicate whether the subject previously failed one ormore challenges, completed one or more challenges, ignored or missed oneor more challenges, how often the subject completed one or morechallenges, how often the subject failed one or more challenges, howwell the subject performed on challenges compared to other subjects, howwell the subject performed on challenges compare to other subjects inthe same group or team, how well the subject performed on challengescompared to past subjects, or the like. This past performanceinformation may be stored and/or organized by the data aggregationmodule 112 shown in FIGS. 1A-1C. This past performance information maybe used, e.g., by the data aggregation module 112 and/or the scoring andprediction module 114 shown in FIGS. 1A-1C, to generate a performancetrend of the subject. The performance trend may be used, e.g., by thescoring and prediction module 114, with the subject's currentperformance in order to determine a readiness score for the subject. Forexample, if the subject is currently performing at a level of 5/10 buthas a positive performance trend, the scoring and prediction module 114may determine that the subject has a readiness score of 6/10. Similarly,if the subject is currently performing at a level of 5/10 but has anegative performance trend, the scoring and prediction module 114 maydetermine that the subject has a readiness score of 4/10.

In some implementations, providing the output includes providing apredicted time that the subject will achieve readiness. This time may beprovided in addition to or in place of a readiness score. The predictedtime may be determined by the scoring and prediction module 114 shown inFIGS. 1A-1C. The predicted time may be calculated using the currentperformance of the subject, the set readiness score or performance goalthat the subject needs to achieve, and the performance trend of thesubject. For example, if the subject's current performance is 5/10, theset readiness score is 8/10, and the subject's performance trendindicates an improvement in performance of 1/10 every month, then thescoring and prediction module 114 may determine and output a time ofthree months for when the subject is predicted to achieve readiness.

In some implementations, providing the output includes providing alikelihood or prediction whether the subject will achieve readiness.This likelihood or prediction may be provided in addition to or in placeof a readiness score and/or a time that the subject will achievereadiness. For example, the likelihood or prediction may be provided asoutput when the scoring and prediction module 114 shown in FIGS. 1A-1Cis unable to determine a time when the subject will achieve readiness,e.g., when the calculated time is infinite due to the subject having acurrent performance under a set readiness score and having a negativeperformance trend. The likelihood or prediction may be determined by thescoring and prediction module 114 shown in FIGS. 1A-1C. The likelihoodor prediction may be calculated using the current performance of thesubject, the set readiness score or performance goal that the subjectneeds to achieve, the performance trend of the subject, and one or moreperformance criteria. For example, if the subject's current performanceis 5/10, the set readiness score is 8/10, the subject's performancetrend indicates an improvement in performance of 1/10 every month, andthe performance criteria requires that the subject achieve readinesswithin two months, then the scoring and prediction module 114 may outputa 20% likelihood or prediction indicating that the subject is unlikelyto achieve readiness in the two-month timeframe.

In some implementations, group readiness criteria refers to readinesscriteria that is applied to every subject in the group. For example, thegroup readiness criteria may require that each person in a group be ableto run two miles in under fourteen minutes, perform twenty pull-ups, andperform fifty pushups within a two month timeline.

In some implementations, the computer system 110 monitors readiness byrepeatedly generating predictions about subjects' readiness and comparesthe predictions with reference values (e.g., thresholds, ranges,targets, milestones, etc.). In some cases, the reference values may beset to measure progress over time relative to trend, such as an averagetrend or progression of readiness acquisition determined from theprogression of multiple subjects. In this manner, if a subject's profilein the database becomes indicative of a change in readiness the computersystem 110 can detect it and provide an appropriate notification, aswell as recommendations of corrective actions to bring the readinessprogression back to the desired level. The computer system can thusdetect when a subject's readiness predictions show a deviation from thetypical trend that has been employed by other subject who havesuccessfully acquired readiness. As noted above, the trends or valuesused for reference can be learned from the data in the database, and socan model many different types of readiness progressions, includingcomplex non-linear trends over time.

To support readiness monitoring, new predictions can be madeperiodically (e.g., at intervals or after a certain time has elapsed) orin response to receiving new data about a subject (e.g., upon receivingupdated sensor data, subject attributes, subject activities, changes totraining plans, etc.). Based on the comparison of the subject'spredicted readiness level and the reference values, alerts ornotifications can be provided, for example, to a device or electronicaddress associated with the subject or a person associated with thesubject (e.g., an administrator, a technician, a coach, a doctor, aresearcher, etc.). For example, if a subject's readiness does not reacha certain milestone (e.g., the readiness declines, the readiness doesnot improve over a period of time, or improves less than a desiredamount), then the computer system 110 can detect this and notifyappropriate devices and users.

In some implementations, the subject is a group of individual subjects.The one or more readiness criteria can include one or more groupreadiness criteria, e.g., representing a level of readiness for thegroup as a whole. Receiving status data can include receiving statusdata indicating activities and/or attributes of individual subjects inthe group. The computer system 110 can generate, based on the activitydata, a readiness score for each of the individual subjects in thegroup. The readiness scores can indicate predictions of the respectivecapabilities of the individual subjects in the group. The computersystem 110 can also generate a group readiness measure that indicates apredicted ability of the group of individual subjects to collectivelysatisfy the one or more group readiness criteria. When the subjectrepresents a group, the output data can indicate (i) a predicted timethat the group will achieve readiness to satisfy the one or more groupreadiness criteria and/or (ii) a prediction whether the group willachieve readiness to satisfy the one or more group readiness criteriagiven one or more constraints.

When new status data is received, e.g., data indicating additionalactivities of the subjects in the group, the computer system 110 cangenerating a new readiness score for each of the subjects in the groupand generate a new group readiness measure. The computer system canprovide additional output data indicating (i) a new predicted time thatthe group will achieve readiness to satisfy the one or more groupreadiness criteria or (ii) a new likelihood or prediction whether thegroup will achieve readiness to satisfy the one or more group readinesscriteria, for example, given one or more constraints. The computersystem 110 may compare the most recent prediction to earlierpredictions, and indicate how readiness has changed over time. Forexample, the computer system 110 can provide output that causes a plotor curve of readiness measures (e.g., scores, readiness times, etc.),lists predictions for different times, or indicates whether predictedreadiness times of scores have increased or decreased.

In some implementations, the computer system 110 may change thecomposition of a group. For example, in response to determining that thegroup readiness measure does not satisfy the threshold, the computersystem 110 may remove from the group one or more subjects determined tohave individual readiness scores that are less than an average ofindividual readiness scores for subject in the group. As anotherexample, the computer system 110 may add to the group one or moresubjects having individual readiness scores that are greater than anaverage of individual readiness scores for subject in the group.

FIG. 13 is a flowchart that illustrates an example process for tailoringtraining to improve readiness. The process 1300 can be performed by oneor more computers, such as the computer system 110 described above.Steps of the process 1300 can be performed by one or more servers, oneor more client devices, or by a combination thereof and/or otherdevices.

The process 1300 includes accessing a database to obtain status datathat indicates attributes and/or activities of a subject (1302). Thiscan include obtaining status data as discussed above with respect tostep 1202 of FIG. 12. As discussed above, the status data can indicatecurrent and former attributes of the subject and a variety of activitiesof the subject over time and results of those activities. The statusdata can include data provided by an electronic device to the one ormore computers over a communication network.

The process 1300 includes generating, based on the activity data, one ormore readiness scores (1304). A readiness score can indicate a level ofcapability of the subject to satisfy one or more readiness criteria. Thereadiness score can indicate a prediction of current readiness or aprediction of future readiness. These readiness scores can be outputs ofmodels, such as machine learning models, statistical models, rule-basedmodels, or other models as discussed above. For example, the one or morereadiness scores can be generated using one or more models such as, forexample, the machine learning models 130, 132, and/or 134 shown in FIG.1C. The readiness score(s) can be obtained for a subject using steps1204, 1206, and 1208 of FIG. 12, as discussed above.

The readiness scores may indicate that the subject is not yet capable ofsatisfying the one or more readiness criteria. For example, thereadiness scores may indicate that the subject is predicted to notsatisfy the one or more readiness criteria within a certain time frame,e.g., at a specific time in the future, given the status data in thedatabase. As discussed with respect to FIGS. 1A-1C, the readiness scoresmay be generated by the scoring and prediction module 114.

The readiness criteria may be set by input provided through a computerinterface, such as a user interface of a client device. The readinesscriteria can be set by a user, for example, an administrator, atechnician, a coach, a counselor, a caregiver, a care provider, aspecialist, an employer, a potential employer, or a supervisor of thesubject. The readiness criteria may make up or be part of thethird-party input 152 shown in FIG. 1B. With respect to FIG. 6, thereadiness criteria may be included in the program data store 638 and/orin the subject outline and content 626 a.

The process 1300 includes accessing data indicating multiple candidateactions for improving capability of the subject to satisfy one or morereadiness criteria (1306). The computer system 110 can access a databasethat includes information describing various different candidate actionsthat can assist subjects to acquire different capabilities. Differentcandidate actions may be better suited for acquiring some capabilitiesthan others. Also, different candidate actions may have better effectsfor subjects having certain attributes than others.

The candidate actions may include interactions of devices, changes inthe types or amounts of resources available, changes in the trainingprogram or training plan for a subject, and so on. For example, theremay be multiple different training programs that use differenttechniques to enhance a person's athletic capability. One set oftraining programs may improve strength, and others may be designed toimprove flexibility, while yet others may be designed to improvebalance, endurance, running speed, swimming ability, and so on. Withineach training program, there may be a combination of elements, such asexercises, classes, practice activities, classes, and other activitiesfor the subject to participate in. The candidate actions can include theassignment of a subject to a training program and/or assignment ofcertain elements within the training program. Additional examples ofcandidate actions may include different medical treatment options, e.g.,physical therapy exercises, medications, surgeries, etc. Other candidateactions may include interactions with mobile devices (e.g., reminders,surveys, media, etc.) or proposed changes in behavior (e.g., changingdiet, sleep habits, and so on).

As additional examples, the candidate actions can include changing afrequency, number, or intensity of planned training activities for thesubject. The candidate actions can include changing an allocation ofresources for the subject. The candidate actions can include changing atype of training for the subject. The candidate actions can includechanging an assignment of an individual to assist the subject inachieving readiness (e.g., assigning a new technician, coach, doctor,manager, etc.). The candidate actions can include providing, for outputby a client device, a notification indicating the level of capability ofthe subject to satisfy one or more readiness criteria that is indicatedby the one or more readiness scores.

The candidate actions can include actions that are manually entered anddesignated for improving specific capabilities. Additionally oralternatively, the candidate actions can be automatically discovered orinferred by the computer system 110 based on the data in the database.Because of the rich set of data that is tracked for many subjects overtime, the computer system 110 can infer the potential effects of manydifferent actions and designate them as candidate actions for improvingdifferent capabilities. For example, the computer system 110 maydetermine, through analysis of the data in the database, that computersystems performed more efficiently if they had been restarted within acertain amount of time, and as a result enumerate restarting a computeras a candidate action to improve readiness. Similarly, for a subjectthat is a person, location data and other data may indicate thatsubjects that spent time outdoors had improved mood or improvedperformance on a task. As a result, a candidate action to cause thesubject's phone to output a message encouraging the subject to take awalk outside may be added as a candidate action.

In the example of FIG. 1C, candidate actions include various trainingoptions indicated in output data 146 (e.g., potential interventions “1”,“2”, “3”, “5”, “6”, and “7”). The candidate actions in that example areshown to indicate the predicted effects of the different options (e.g.,achieving readiness “12% faster” or “4% faster”).

The process 1300 includes selecting a subset of the candidate actionsfor the subject based on the one or more readiness scores (1308). Thesubset can be a proper subset, e.g., fewer than all of the candidateactions evaluated. Various different techniques can be used to selectcandidate actions to recommend or carry out for a subject. For example,the set of available actions may be filtered according to those that arepredicted to improve readiness with respect to the readiness criteriafor the subject. In other words, if the readiness criteria includemeasures of strength or speed, candidate actions that are not predictedto improve those capabilities are not selected. Similarly, the set ofcandidate actions can be filtered to a subset by evaluating themagnitude of predicted improvement, the resources required for theactions, or both, so that only actions that provide at least a minimumlevel of predicted improvement and/or require less than a maximum amountof resources are selected.

Whether any candidate actions are selected to be carried out orrecommended can be based on the readiness scores. The one or morereadiness scores indicate a predicted current or future level ofreadiness, for example, as a prediction of when or whether the subjectwill achieve readiness, or as a score along a scale for assessing levelsof capability. Actions to change the training or preparation of asubject can be selected in response to the readiness scores indicatingthat improvement in readiness is needed. For example, the scores canindicate whether intervention is needed for the subject to reach adesired level of readiness, as well as the type and extent (e.g.,magnitude, frequency, or intensity) of intervention needed.

The computer system 110 may use the one or more readiness scores todetermine whether the subject currently meets or a target level ofreadiness or is projected to do so in the future. When the computersystem 110 determines that scores do not meet the target level, thecomputer system 110 can select a subset of the candidate actions for thesubject in order to improve the readiness of the subject. Thus, based onthe level of readiness indicated by the readiness scores, the computersystem 110 can select a type and amount of candidate actions that arepredicted to bring the subject's readiness to or closer to a desiredreadiness level or trend of acquiring readiness.

For example, the readiness scores for the subject can be compared withreference scores, and the comparison may indicate that the subject doesnot yet satisfy the one or more readiness criteria or that the subjectis predicted to not satisfy the readiness criteria at a time in thefuture. Similarly, the trend or progression of the subject's readinessscores over time may be compared with the trends or progressions ofsubjects that successfully achieved readiness and subjects that did not.In other words, the scores for a subject at a particular stage inacquiring readiness (e.g., after 1 month of training) may be comparedwith scores representing other subjects at the same stage (e.g., whenthe other subjects had completed 1 month of training).

As discussed below, a customized subset of actions can selected for eachsubject, taking into account the subject's attributes, the subject'shistory of activities, the subject's current capabilities and predictedfuture capabilities, the subject's planned future activities, theparticular set of readiness criteria for the subject, and so on. Theselection can also take into account the predicted improvement toreadiness that the respective actions provide and the amount ofimprovement in readiness that the particular subject needs in order toreach a desired readiness level or readiness progression.

The computer system 110 can access data indicating how the respectivecandidate actions are predicted to affect the readiness of the subjectto satisfy the one or more readiness criteria. The indications can bestored in association with the data indicating the candidate actions.For example, in a manner similar to the way the table 400 in FIG. 4shows the potential effects of different genes on capabilities and themagnitude of those effects, data can also be stored to indicate themanner and magnitude of each of the candidate actions (e.g., trainingoptions or other interventions) on different capabilities. The valuesindicating the manner and magnitude of these effects can be generatedfrom the data in the database, which shows the longitudinal progressionof readiness of various subjects over time. The computer system 110 canidentify instances of the different candidate actions among the data inthe database, and then evaluate the relative effect of those actions onthe capabilities of the subjects at one or more times in the future.

In some implementations, the data indicating the predicted effects ofcandidate actions on different capabilities are expressed as scores. Forexample, a particular activity, such as running three miles, may havemultiples scores, each representing how the activity is expected toaffect capability or physical state of the subject, e.g., running speed,running endurance, balance, heart health, etc. These scores may bepredetermined for the various candidate actions. In someimplementations, the computer system 110 can adjust or generate thescores for candidate actions in a customized manner for individualsubjects. For example, the computer system 110 can generate the scorescan take into account the particular attributes of the subject, forexample, by weighting the information in the database about othersubjects according to their similarity to attributes of the subjectbeing evaluated. In a similar manner, the scores can be customized toaccount for the history of activities of the subject, the progression orchange in attributes of the subject over time, and other factors.

The process 1300 includes providing one or more outputs based on theselection of the subset of candidate actions for the subject (1310). Insome cases, the output causes one or more selected actions to beperformed or causes an indication of the selected actions to bepresented on a user interface.

The output can include output data configured to cause one or more ofthe actions in the selected subset to be performed by the computersystem 110 or another system. For example, the computer system 110 maychange to a training plan for the subject to include the selectedactions, or may send a command to another system to do so. Variousactions such as initiating communications with a subject, sending acommand to a device, changing data collection and monitoring, oraltering software behavior can be performed directly by the computersystem 110.

In some implementations, the computer system 110 may initiate at leastsome types of actions automatically when they are selected as beingappropriate for the subject. For example, for actions within a certaingroup or actions that require less than a maximum level of resources,the computer system 110 may automatically adjust training plans andinteractions with the subject to include those actions if predicted tobe appropriate for the subject. The computer system 110 may determine toimplement the selected actions that it is authorized to carry out, ormay apply further evaluation criteria to filter the actions further. Forexample, the computer system 110 may evaluate the predictedeffectiveness and resource requirements of the actions, and determine toimplement only actions that provide at least a minimum level of increaseto readiness predictions and/or involve less than a maximum level ofresources.

As another example, the computer system 110 can cause an indication oneor more of the actions in the selected subset to be presented on a userinterface of the computer system 110 or another system. For example, thecomputer system 110 can send data causing a user interface of a deviceto indicate the selected actions as recommendations or options toimprove readiness. The indications may be provided for approval orconfirmation by a user before they are carried out. The indications canbe provided to a device of an individual associated with the subject,for example, a manager, technician, coach, counselor, caregiver, careprovider, doctor, specialist, employer, potential employer, orsupervisor. Examples include the device 208 shown in FIG. 2A or thedevice 308 shown in FIG. 3. The output may be presented on a userinterface of a device associated with the subject, e.g., the device 204shown in FIG. 2A or the device 304 shown in FIG. 3.

In some implementations, the computer system 110 uses the selectedactions to change a plan of future activities to increase readiness ofthe subject to satisfy the one or more readiness criteria. The computersystem 110 can provide a recommendation, for presentation on a userinterface of a client device, of one or more of the actions in theselected subset. For example, the computer system 110 can cause to bepresented the top N candidate actions having the highest predictedimprovement to readiness of the subject, where N is an integer.

In some implementations, the computer system 110 provides one or moreinteractive user interface controls configured to initiate one or moreof the actions in the subset in response to user interaction with theone or more interactive user interface controls. For example, a buttonor other control can be provided for each option, and the correspondingaction can be performed when a user selects the corresponding control.

In some implementations, the output data includes one or more scoresindicative of a predicted change to the readiness of the subject tosatisfy the one or more readiness criteria. IN addition, or as analternative, the output data can provide visualization data toillustrate a timeline or trend indicating a progression of readiness ofthe subject to satisfy the one or more readiness criteria if one or moreof the selected actions are performed.

The user interface can be configured to show the predicted effects ofselecting to implement different actions or combinations of the actionsfor the subject. For example, different predicted readiness dates can beshown for each of the different options. As another example, scores orother indicators can indicate the amount of improvement expected (e.g.,an amount of time reduced before reaching readiness, a percentage ofresources saved, an amount that capability is increased, etc.). Asanother example, the different progressions of readiness over time fordifferent actions can be presented in a visualization on a userinterface, such as in a chart, graph, table, timeline, etc. For example,a chart can be provided that shows different curves representing theprojected progression of readiness scores for the subject in the futureif different actions are implemented, along with a curve showing theexpected progression of readiness of the subject if none of the optionsare selected.

FIG. 14 is a flowchart that illustrates an example process forpredicting readiness. The process 1400 can be performed by one or morecomputers, such as using the computer system 110 described above. Stepsof the process 1400 can be performed by one or more servers, one or moreclient devices, or by a combination thereof and/or other devices.

The computer system 110 can be configured to incorporate and interfacewith any of various third-party models and analysis techniques. Forexample, the computer system 110 may provide an interface for a user toadd, download, or connect models or analysis software to the computersystem 110, to provide additional options for the computer system 110 touse. Of course, the computer system 110 may additionally oralternatively generate and train models based on data in the database125.

The process 1400 includes receiving data indicating attributes oractivities of a subject (1402). This can include obtaining data from adatabase, as discussed for step 1202 of FIG. 12. Data can be receivedthrough a variety of devices, sensors, user inputs, and so on. Withrespect to FIG. 1B, the data may include sensor data collected by one ormore of the sensing devices 104 a-104 c and sent to the computer system110. The sensor data may include data collected while the subject, e.g.,person, is engaged in a particular activity indicated in the receiveddata. The activity data may include data self-reported by the subject.

The process 1400 includes accessing data indicating multiple differentanalysis options to assess readiness with respect to one or morereadiness criteria (1404). In some cases, the analysis options assessthe potential of a subject to achieve readiness by being capable ofsatisfying the one or more readiness criteria. As discussed above,assessing readiness may involve predicting current readiness, predictingfuture readiness, predicting a time readiness will be achieved,predicting whether readiness will be achieved by a certain time, and soon. Multiple analysis options can be available and identified for eachof these different types of predictions.

Each of the multiple analysis techniques can use example data indicating(i) activities or attributes of other subjects and (ii) outcomes for theother subjects with respect to at least one of the one or more readinesscriteria. The analysis techniques can predict readiness of subjects tosatisfy one or more readiness criteria based on progressions ofreadiness of the other subjects over time, as indicated in the exampledata from the database.

The different analysis options can be different analysis techniques, forexample, multiple different models, that are all configured to make asame type of prediction, thus allowing the different options to generatedifferent predicted values that are comparable (e.g., represent the sameitem being predict). For example, there may be different models forpredicting future readiness probability for a particular set ofreadiness criteria, where the models take into account differentcombinations of input data, have been trained using different trainingdata, use different assumptions or inferences, are configured to predictwith different accuracy or variance characteristics, and so on. Thecomputer system 110 may have multiple of these options available, sothat the computer system 110 or a user can select one or moreappropriate options from the set. In some cases, the computer system 110generates predictions of the same type using different analysis options(e.g., different models, different constraints, input data representingdifferent granularity or time ranges, etc.), and provides the multiplepredictions for presentation.

In some implementations, the multiple analysis options comprise modelstrained based on different sets of training data, using differenttraining parameters, or using different training techniques. The modelsmay include models with different structures (e.g., neural networks withdifferent sizes, different numbers of layers, different types of layers,different sizes of layers, etc.). In some implementations, the multipleoptions include using models of different types. For example, the modelscan include two or more different types such as a neural network, asupport vector machine, a classifier, a regression model, areinforcement learning model, a clustering model, a decision tree, arandom forest model, a genetic algorithm, a Bayesian model, and aGaussian mixture model. At least some of the multiple models can bemodels trained to make predictions of readiness with different levels ofvariance, for example, to reflect different bias/variance tradeoffs. Atleast some of the models can be trained to make predictions based ondiffering combinations of input features representing different sets ofinformation about the subject.

The process 1400 includes selecting one of the multiple differentmachine learning techniques based on a user input (1406). Once themultiple analysis options are identified by the computer system 110, thecomputer system 110 may provide data for the different options to beindicated to a user, e.g., on a user interface. For example, based onthe accessed data indicating the multiple different analysis techniques,the computer system 110 can provide, for display on a user interface, anindication of each of the multiple different analysis techniques. Thecomputer system 110 may first filter the list of options to thoseapplicable to the current data set, for example, the particular subject,readiness criteria, prediction type to be generated, set of collecteddata in the database, etc. corresponding to the current task or viewcorresponding to the user interface. The computer system 110 can receivedata indicating user interaction with the user interface that selectsone of the multiple different analysis techniques. For example, the usermay interact with buttons, checkboxes, or other controls to indicatewhich of the analysis options the user selects to be used. The computersystem 110 can then select and use the model(s) that the user inputindicated should be used.

Other types of user input may be used by the computer system 110 toselect the appropriate analysis technique. For example, without userinput that directly selects an analysis technique or model, the computersystem 110 may receive user input that indicates other user preferences.The computer system 110 may provide data causing a user interface toprovide controls to indicate preferences such as desired speed ofobtaining a prediction, a risk tolerance (e.g., an extent thatpredictions should bias against false positives, such as overestimatesof readiness, or false negatives, such as underestimating readiness),whether any restrictions on the types of data used in the predictionshould be taken into account, and so on. The computer system 110 canthen select, from among the available analysis options, the option thatbest suits the user preferences. To aid in this selection, the computersystem 110 can generate and store metrics that characterize eachanalysis option. For example, models can be evaluated and validated aspart of training, and the accuracy, precision, computational cost,efficiency, and other parameters can be computed and stored as a profilefor each model. The computer system 110 can then select the model(s)having a profile that most closely matches the characteristics indicatedby user preferences.

For example, with respect to FIG. 1C, selecting one of the multipledifferent machine learning techniques based on a user input may includeselecting one of the machine learning models 130, 132, and 134 based onthe user input. For example, with respect to FIG. 2B, selecting one ofthe multiple different machine learning techniques based on a user inputmay include selecting a machine learning or model to generate aprediction for CRT, FRP, and/or ACS based on the user input. The userinput may be made by a user of the system. The user input may be madeby, for example, a coach, counselor, caregiver, care provider,specialist, employer, potential employer, supervisor, or other userassociated with the subject. In some cases, the user input may be madeby the subject.

The process 1400 includes generating a measure of predicted readiness ofthe subject with respect to the one or more readiness criteria (1408).This can involve generating feature scores and generating a predictionusing a model for the selected analysis option, using techniquesdiscussed above with respect to steps 1204, 1206, and/or 1208 of FIG.12. The measure of predicted readiness can be determined based on any ofthe factors discussed above, such as current attributes of the subject,previous or former attributes of the subject, current or previousactivities of the subject (e.g., activities assigned to the subject toincrease a capability of the subject, or other activities), a planned orscheduled activities of the subject (e.g., a training plan indicatingactivities selected to increase a capability of the subject), and so on.

The measure of predicted readiness of the subject can be generatedaccording to the selected machine learning technique. The measure ofpredicted readiness may be a measure of predicted readiness of thesubject to satisfy one or more readiness criteria. The feature data mayindicate attributes of the person such as, for example, physiologicaland psychological attributes. With respect to FIG. 1B, the feature datamay be derived using the computer system 110, e.g., using the dataaggregation module 112 to extract and/or organize the activity data. Thefeature data may include all or part of the aggregated data 122 shown inFIG. 1B. The feature data may be or include feature scores.

In some implementations, multiple measures of predicted readiness of thesubject are generated prior to the selecting one of the multipledifferent machine learning techniques based on a user input. Forexample, each measure of predicted readiness generated may be generatedusing a different machine learning techniques. These generated measuresmay be sent to the subject and/or to a coach, counselor, caregiver, careprovider, specialist, employer, potential employer, supervisor of thesubject. These generated measures and/or visual representations of thegenerated measures may be presented on a user interface of a devicebelonging to the subject, and/or a user interface of a device belongingto a coach, counselor, caregiver, care provider, specialist, employer,potential employer, supervisor of the subject. Accordingly, the subject,and/or a coach, counselor, caregiver, care provider, specialist,employer, potential employer, supervisor of the subject may use thesemultiple measures and/or visual representations of the generatedmeasures in selecting one of the multiple different machine learningtechniques. As an example, the output 140 shown in FIG. 1B may bepresented on a device belonging to the subject, and/or a devicebelonging to another person associated with the subject, e.g., a coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, or supervisor.

The process 1400 includes providing output data based on the selectedanalysis technique (1410). The output data can update the user interfacebased on the measure of predicted readiness of the subject. In addition,or as an alternative, the output data can cause an interaction with oneor more devices based on the measure of predicted readiness of thesubject. For example, the output can include output as discussed abovewith respect to step 1210 of FIG. 12 or step 1310 of FIG. 13.

In some implementations, the computer system 110 provides, for displayon a user interface, data indicating different predictions of readinessfor a subject, where the different predictions were each generated usinga different one of the multiple different analysis techniques. Forexample, as shown in FIG. 1C, predictions generated using differentmodels (e.g., “Model A,” “Model B,” and “Model C”) are generated andindicated. In the FIG. 1C example, measures of readiness are provided asscores, as well as a curve or trend line showing the expectedprogression of acquiring readiness over time. The computer system 110also evaluates the suitability of the models and provides an indicationof this analysis to assist the user, e.g., indications to “recommend”using Model B, or “do not recommend” for Model C.

To provide the output data, the computer system 110 may provide one ormore of the following. For example, the computer system 110 may providean indicator of the measure of predicted readiness for display on a userinterface. The computer system 110 may provide visualization data,generated using the measure of predicted readiness, for illustrating atimeline or trend of predicted readiness of the subject over time. Thecomputer system 110 may provide a recommendation, determined based onthe measure of predicted readiness, of an action to improve oraccelerate acquisition of readiness of the subject to satisfy the one ormore readiness criteria. The computer system 110 may provide one or moreinteractive user interface controls to alter one or more planned actionsfor assisting the subject to achieve readiness to satisfy the one ormore readiness criteria. The computer system 110 may provide one or moreinteractive user interface controls to select from among multipleoptions for assisting the subject to achieve readiness to satisfy theone or more readiness criteria. The computer system 110 may provide anindication of a predicted change in readiness of the subject, relativeto a level of readiness indicated by the measure of predicted readiness,for each of one or more actions with respect to the subject.

In some implementations, the output data causes a device associated withthe subject to initiate interactions that are selected based on themeasure of predicted readiness of the subject. For example, with respectto FIG. 10, a program may be selected for the subject based on themeasure of predicted readiness of the subject that includes a list ofchallenges (e.g., hip stability drill, military movement drill, run,etc.). The interactions may also include requesting feedback from thesubject after a challenge is complete, after a certain amount of timehas passed (e.g., one day, one week, one month, etc.), after a milestonehas been reached (e.g., after subject has reached readiness, subject'sperformance trend indicates that the subject is expected to reachreadiness, subject's performance trend indicates that the subject isexpected to reach readiness by a certain point in time, etc.), after thesubject has completed a particular percentage of the program (e.g., 25%,50%, 75%, 100%), or the like. This feedback may include, for example, alevel of pain, and/or a feeling (e.g., good, bad, neutral, etc.). Thedevice may be, for example, the device 204 shown in FIG. 2A, or thedevice 304 shown in FIG. 3.

FIG. 15 is a flowchart that illustrates an example process forpredicting readiness of a group or a team. The process 1500 can beperformed by one or more computers, such as using the computer system110 described above. Steps of the process 1500 can be performed by oneor more servers, one or more client devices, or by a combination thereofand/or other devices.

The process 1500 includes receiving data designating a group comprisingmultiple individual subjects (1502). For example, the computer system110 can access stored records that define different groups or teams, byindicating the assignment of different individual subjects to groups. Asanother example, the computer system 110 may receive an identifier for agroup of interest, for example, through user input to a user interfacethat indicates a particular team of people for which a prediction is tobe generated. The computer system 110 can access stored records defininggroup membership to determine the members of the indicated group.

As an example, a group could refer to a group of components of a deviceor a group of devices in a system, which each element has individualparameters and capabilities but the function of the entire grouptogether is important. As another example, a group could be a sportsteam, a military unit, a department of an organization, or other groupof people whose individual capabilities contribute to and affect theeffectiveness of the group as a whole.

The process 1500 includes receiving status data indicating attributesand/or activities of subjects in the group (1504). This data may beobtained for individual subjects in the group (e.g., each subject in thegroup) and/or for the group as a whole. The data can include dataobtained as discussed in step 1202 of FIG. 12, for any and all of theindividual subjects and/or for the group itself. As discussed withrespect to FIG. 1B, the data may include sensor data, data that isself-reported by the subjects, and so on.

The process 1500 includes generating a group readiness measure based onthe status data (1504). The group readiness measure can indicate apredicted ability of the group to satisfy one or more group readinesscriteria, e.g., at a current or future time. The group readinesscriteria can be similar to the readiness criteria discussed above, butapplies to the group as a whole to determine whether the group achievesa level of capability. For example, the group readiness criteria mayspecify that each of the individual members respectively meets someindividual readiness criteria. As another example, the group readinesscriteria may specify that the group should be capable of performingcertain tasks, though not every member is required to have capability toperform all the tasks. The group readiness criteria may be a function ofindividual readiness scores (e.g., predictions).

The group readiness criteria may specify, for example, at least one of aphysical state of the members of the group, a functional state of themembers of the group, or a level of capability of the members of thegroup to perform a task.

As part of this process, information for each of the subjects in thegroup can be used and processed using one or more models. This mayinvolve generating a prediction using the techniques of steps 1204,1206, and/or 1208 of FIG. 12 for the group.

The group readiness measure may be generated using the one or moremodels, for example, at least one of a neural network, a support vectormachine, a classifier, a regression model, a clustering model, adecision tree, a random forest model, a genetic algorithm, a Bayesianmodel, or a Gaussian mixture model. Machine learning models, statisticalmodels, rule-based models, and other types of models can be used. Theone or more models can be models trained based on training dataindicating (i) activities or attributes of other subjects and (ii)outcomes for the other subjects with respect to corresponding readinesscriteria. The training can use the respective progressions of readinessof the other subjects over time to configure the one or more models topredict readiness of subjects to satisfy one or more readiness criteria(e.g., individual readiness criteria). In a similar manner, theinformation about how groups have progressed over time can be used totrain models to predict how a group will progress toward readiness tosatisfy one or more group readiness criteria. As with the other modelsdiscussed herein, the models can be trained for specific readinesscriteria, or may be trained more generally with the specific readinesscriteria being input to or accessed by the model at the time predictionsare made.

Generating the group readiness measure may include generatingpredictions for individual members of the group, with respect toindividual readiness criteria, and then combining individual results(e.g., averaging, applying a weighted combination, or otherwise applyinga function to the individual scores) to determine the group readinessmeasure. The group readiness measure may indicate a prediction, such asa predicted time that the group will achieve readiness to satisfy theone or more group readiness criteria or a prediction whether the groupwill achieve readiness to satisfy the one or more group readinesscriteria.

In some implementations, a model may be generated or trained to predictgroup readiness measures based on information about the constituentmembers of the group. The model may be configured to receive inputfeature data about individual members and/or data indicating results ofpredictions of other models, such as readiness prediction values frommodels that assess individual capabilities of the members of the group.Thus, the group readiness measure may be, or may be based on, outputfrom a model configured for this purpose.

In some implementations, generating the group readiness measure caninclude generating an individual readiness score for each of themultiple subjects in a group. The individual readiness scores indicatepredictions of the respective capabilities of the subjects in the group.The group readiness measure can be generated based on the individualreadiness scores for the subjects in the group. The individual readinessscores can respectively indicate predicted readiness of individualsubjects to satisfy one or more individual readiness criteria. Theindividual readiness criteria can be different from the group readinesscriteria.

To generate individual readiness scores, the computer system 110 can,for each of the multiple subjects: (1) provide, to one or more models, aset of feature scores for the subject derived from status data for thesubject obtained from a database of attributes or activities ofsubjects; and (2) receive an output that is generated for the subject inresponse to the one or more models receiving the set of feature scoresfor the subject, where the individual readiness score for the subject isbased on the output.

As an example, the group readiness measure can be an aggregate measureof the individual readiness scores for the subjects in the group, e.g.,an average (e.g., mean, median, mode, etc.), a sum of the individualreadiness scores, a maximum or minimum value of the individual readinessscores, a weighted combination, or other function of the individualreadiness scores. As another example, to generate the group readinessmeasure, the computer system 110 can provide the individual readinessscores to one or more models trained to generate a group readinessmeasure based on examples of group readiness measures and correspondingindividual readiness scores for members of the group. The computersystem 110 can determine the group readiness measure based on theprocessing of the one or more models performed in response to receivingthe individual readiness scores.

The readiness scores can indicate predictions of the respectivecapabilities of the members in the group. For example, the readinessscores can indicate a level of capability of each of the subjects tosatisfy one or more readiness criteria. With respect to FIGS. 1A-1C, thereadiness scores may be generated by the scoring and prediction module114. The readiness scores can also indicate that one or more of thesubjects are not yet capable of satisfying the one or more readinesscriteria. The readiness criteria may be set by, for example, a coach,counselor, caregiver, care provider, specialist, employer, potentialemployer, supervisor of the subjects. The readiness criteria may make upor be part of the third-party input 152 shown in FIG. 1A. With respectto FIG. 6, the readiness criteria may be included in the program datastore 638 and/or in the subject outline and content 626 a. The readinessscores can be generated using one or more machine learning models suchas, for example, the machine learning models 130, 132, and/or 134 shownin FIG. 1C.

The process 1500 includes providing output data based on the groupreadiness measure (1508). The output data can be configured to, forexample, alter a user interface of a device or alter interaction of adevice with one or more of the subjects in the group.

In some implementations, the computer system 110 uses the group measureto determine whether the group has achieve or is appropriatelyprogressing toward readiness. The computer system 110 can use theprocess 1300 of FIG. 13 to select, as well as indicate and/or implement,actions to improve the capabilities of the group as a whole orindividual members of the group. Thus, any of various interactionspredicted to improve readiness of individuals or the group as a wholecan be instructed to be performed by devices. For example, changes todata collection processes, alerts, reminders, forms, surveys, and so oncan be identified and made in response to determining that the groupreadiness score does not provide a desired level of readiness or levelof confidence that readiness will be achieved.

In some implementations, the computer system 110 determines arecommended change to the group to improve the readiness of the group.For example, the computer system 110 may determine to add or removemembers from groups, to reassign members to different roles withingroups, or to make other changes. The computer system 110 may identifychanges to groups that would increase the total number of groups thatmeet the group readiness criteria and recommend or implement thechanges. As an example, the computer system 110 may determine, based onreadiness scores for two groups, that the first group has achievedreadiness but the second group has not. The computer system 110 mayfurther determine, based on individual readiness scores for members ofthe groups, that switching a high-performing member of the first groupwith a lower-performing member of the second group would allow bothgroups to reach readiness, and in response either recommend the changeor implement the change.

The output data provided can indicate a prediction, such as (i) apredicted time that the group will achieve readiness or (ii) aprediction (e.g., classification or likelihood) whether the group willachieve readiness given one or more constraints. The group achievingreadiness may include the group achieving readiness to satisfy the oneor more group readiness criteria. The readiness criteria may be set by,for example, a coach, counselor, caregiver, care provider, specialist,employer, potential employer, supervisor of the subjects. The readinesscriteria may make up or be part of the third-party input 152 shown inFIG. 1A. With respect to FIG. 6, the readiness criteria may be includedin the program data store 638 and/or in the subject outline and content626 a. With respect to FIGS. 1A-1C, the predicted time that the groupwill achieve readiness, or the likelihood or prediction whether thegroup will achieve readiness given one or more constraints may begenerated by the scoring and prediction module 114.

As examples, the output data can include one or more of: a scoreindicative of the prediction for display on a user interface; anindicator of the prediction for display on a user interface;visualization data, generated using the prediction, for illustrating atimeline or trend of predicted readiness of the group or one or moresubjects in the group over time; a recommendation, determined based onthe prediction, of an action to improve or accelerate acquisition ofreadiness of the group to satisfy the one or more group readinesscriteria; a recommendation, determined based on the prediction, of anaction to improve or accelerate acquisition of readiness of one or moresubjects in the group to satisfy one or more individual readinesscriteria; one or more interactive user interface controls to alter oneor more planned actions for assisting the group to achieve readiness tosatisfy the one or more group readiness criteria; one or moreinteractive user interface controls to select from among multipleoptions for assisting the group to achieve readiness to satisfy the oneor more readiness criteria; or an indication of a predicted change inreadiness of the group, relative to a level of readiness indicated bythe prediction, for each of one or more actions with respect to thesubject.

The processes of FIGS. 12-15 can be used to evaluate many differentaspects of performance, including different aspect of human performance.This can include readiness of a person to face challenging situationsand stress in a job role or other situation. To gauge these risks, thecomputer system 110 can predict readiness in holistic manner, acrossvarious different dimensions. For example, fitness for a task or rolemay involve components that are physical, emotional, cognitive,intellectual, spiritual, financial, and ethical. For example, a task mayhave physical requirements for a subject to perform, but may also havemental or emotional demands as well. Some tasks may require a person tobe ethically or financially prepared. For example, a person in a rolehandling money may need to be financially prepared to that embezzlingmoney is not a serious temptation, or a person working in a negotiationmay need to be ethically and financially prepared to refuse bribes. Thereadiness criteria that are set for a task or role can includecomponents or criteria for any and all of the different dimensions ofreadiness that are applicable to a desired state of readiness.

In some implementations, the computer system 110 can be used to predictand enhance readiness as part of a total force fitness (TFF) program.For example, the computer system 110 can be used to support individualsand groups of people in understanding their levels of readiness andtheir trends and progress, and can provide personalized recommendationsabout how to improve. The predictions and recommendations can be basedon various areas, including psychological, behavioral, spiritual,social, physical, medical and dental, nutritional, and environmentaldomains. Predictions of readiness and recommendations for improvementcan be made for individuals, military units of different types or sizes,classes or groups of people undergoing training, teams for particulartasks or purposes, and so on. The predictions can recommendations can beto achieve readiness to carry out of job tasks and/or to have thegeneral physical and mental state needed for wellness that is not tiedto specific job tasks. These predictions and recommendations can be madebased on models developed from data sets describing the attributes andactivities of soldiers and other personnel, using the techniquesdiscussed herein.

The types of predictions that can be made by the computer system 110include predictions of how long a state of readiness will be maintainedby a subject. In other words, the computer system 110 can predict notonly when and whether a subject will achieve readiness, but additionallyor alternatively how long the state of readiness (e.g., attributes orabilities that satisfy the readiness criteria) will persist. This can bepredicted as, for example, a predicted duration (e.g., an amount oftime, such as a month, a year, etc. following the achievement ofreadiness), a specific date or time when readiness may be lost (e.g.,readiness is expected until some predicted date or time in the future),or as a measure (e.g., a binary classification, confidence score,probability measure, etc.) whether readiness will be maintained at somefuture time. Similarly, the amount of variation in readiness over time(e.g., a variance or range that readiness is expected to fluctuatewithin) can be predicted.

In general, models can be trained to predict maintenance of readiness inthe same manner that models are trained to predict acquisition ofreadiness, taking into account the same factors discussed above.Similarly, models can be trained to predict the effects of differentcandidate actions or interventions and their effects on maintainingreadiness, so interventions can be selected to maintain readiness, justas models are trained and used to predict and select actions forreaching readiness in the first instance.

The predictions for maintaining readiness can be conditioned onpredicted different activities involving the subject, which may or maynot include proposed steps or plans designed to facilitate maintainingreadiness. For example, the prediction may take into account a readinessmaintenance plan for regular steps for physical training, practicing askill, performing maintenance on equipment, periodic tests, and so on.Different candidate maintenance plans can be defined, and predictionscan be made for each to indicate the different levels of readiness thatmay be maintained with different plans. Similarly, the expected tasks orworkload of the subject can be taken into account. For example, asubject that is overloaded with a task may decline in readiness forbeing overworked. On the other hand, a subject that does not use orpractice abilities required for readiness may decline in readiness forlack of practice or other factors. The computer system 110 can analyzedata indicating the progression of readiness of subjects after theirinitial achievement of readiness according to readiness criteria, alongwith their activities and attributes over time, to determine howdifferent everyday activities, work tasks, and interventions (e.g., suchas training or maintenance actions) affect the maintenance of readiness.This can allow the computer system 110 to predict when additionalactions, such as training or testing, would be needed or beneficial fordifferent subjects. The computer system 110 may use machine learningmodels to learn, for example, the relationships between differentsubject attributes and activities and the eventual readiness maintenanceprofiles over time, and actions that best maintain readiness.

To make these predictions, one or more models can be trained to modelthe characteristics that affect readiness changes, once readiness hasbeen achieved, for different subjects and different readiness criteria(e.g., different types of capabilities). The models can use examples ofsubjects whose readiness declines as well as subject whose readiness ismaintained (e.g., kept the same or improved) over time. This trainingprocess can incorporate into the model(s) data indicative of the factorsthat led to maintenance of readiness by various subjects, the factorsthat led to decreasing readiness, and so on, along with the extent ofthe effects (e.g., magnitude, amount of variability, duration or latencyof effects, etc.). Thus, the models can learn the typical progressionsof readiness both in the process of achieving readiness in the firstinstance and for maintaining readiness thereafter. Indeed, some actionsor training steps may be assessed for their impact on both readinessacquisition and readiness maintenance. For example, some trainingactions may promote achieving readiness quickly, but in a manner thatreadiness is not as stable or prolonged. This may be taken into accountso the computer system 110 prioritizes other actions that may lead toslower but more lasting acquisition of readiness in recommendations.Similarly, some actions for maintaining or acquiring readiness for oneset of criteria (e.g., for a first task or skill) may provide benefitsfor achieving readiness for another set of criteria (e.g., for a secondtask or skill). Thus, the computer system 110 can give priority toactions that maintain the readiness for one set of criteria and whichalso maintain and improve readiness for one or more other set ofreadiness criteria that are not applicable or required at the currenttime, such as criteria for an potential future job role, another task,or a new capability that the subject may need to meet in the future.

The data collected by the computer system 110 and used in any of theexamples and implementations discussed above can include a variety ofinformation from a variety of sources. Data can be collected forcategories representing a variety of individual, community, or publichealth conditions and behaviors. This data can include attributes thatare biological, physical or physiological, mental, emotional,environmental, or social. The collected data can include biologicalattributes, such as genetic makeup, genomics, family history, sensoryabilities (e.g., ability to see, perception of light and dark,perception of color, extent of ability to smell, ability to touch andsensitivity, ability to hear and sensitivity, etc.). These may reflectbiological factors that a person cannot control. The collected data caninclude physical or physiological attributes, e.g., weight, muscle mass,heart rate, sleep, nutrition, exercise, lung capacity, brain activity,etc. Some physical attributes may result from the impact of lifestylechoices or things that a person can control. The collected data caninclude mental attributes, such as interpretation of brain relatedsignals, indications of chemical imbalances, education levels, resultsof mental tests, etc. The collected data can include emotionalattributes, such as interpretation of self-reported data, or classifiedaudio or video related data that suggests individual responses tostimuli. The collected data can include environmental data, such aslocation data, air quality, audible noise, visual noise, temperature,humidity, movement (and potentially effects of movement such as motionsickness, etc. The collected data can include social attributes, such aswhether a subject is socially engaged, exhibits social avoidance,experiences the impact of acceptance or responsiveness emotionally, andso on.

The data collected and used by the computer system 110 (e.g., togenerate feature values, to train models, to identify and select actionsto improve readiness, etc.) can include various other types of dataincluding:

-   -   Lab and diagnostic data (e.g., assay data, blood test results,        tissue sample results, endocrine panel results);    -   Omics data (e.g., data relating to genomics, proteomics,        pharmacogenomics, epigenomics, metabolomics, biointeractomics,        interactomics, lifeomics, calciomics, chemogenomics, foodomics,        lipidomics, metabolomics, bionomics, econogenomics,        connectomics, culturomics, cytogenomics, fermentanomics,        fluxomics, metagenomics, metabonomics, metallomics,        O-glcNAcomics, glycomics, glycoproteomics,        glycosaminoglycanomics, immunoproteomics, ionomics, materiomics,        metalloproteomics, metaproteogenomics, metaproteomics,        metatranscriptomics, metronomics, microbiomics, microeconomics,        microgenomics, microproteomics, miRomics, mitogenomics,        mitoproteomics, mobilomics, morphomics, nanoproteomics,        neuroeconomics, neurogenomics, neuromics, neuropeptidomics,        neuroproteomics, nitroproteomics, nutrigenomics,        nutrimetabonomics, oncogenomics, orthoproteomics, pangenomics,        peptidomics, pharmacoeconomics, pharmacometabolomics,        pharmacoproteomics, pharmaeconomics, phenomics,        phospholipidomics, phosphoproteomics, phylogenomics,        phylotranscriptomics, phytomics, postgenomics, proteogenomics,        proteomics, radiogenomics, rehabilomics, retrophylogenomics,        secretomics, surfaceomics, surfomics, toxicogenomics,        toxicometabolomics, toxicoproteomics, transcriptomics,        vaccinomics, variomics, venomics, antivenomics, agrigenomics,        aquaphotomics);    -   Biologically sampled data (e.g., data describing blood, urine,        saliva, breath sample, skin scrape, hormone levels, ketones,        glucose levels, breathalyzer, DNA, perspiration, and other        biological samples and derived data);    -   Cardiac-related biodata (e.g., data from ECG/EKG monitors, heart        rate monitors, blood pressure monitors);    -   Respiratory-related biodata (e.g. data from spirometers, pulse        oximeters);    -   Neurological-related biodata (e.g. data from EEG monitors);    -   Behavior data (e.g. movement patterns, gait, social avoidance);    -   Drug data (e.g., prescription information, pharmacological        data);    -   Substance use data (e.g., alcohol, medication, insulin,        recreational drugs, tobacco);    -   Sleep data (e.g., motion data, heart rate data, body        temperature, perspiration, breathing data, ambient light,        ambient sound, ambient temperature);    -   Exercise data (e.g. performance data, distance covered,        activity, VO2 Max),    -   Physical activity data (e.g., step counts, heart rate, flights        climbed, altitude, other data from fitness trackers);    -   Mood data (e.g., happiness, depression, PHQ9, BMIS data and        other scales/reporting mechanism);    -   Positioning and location data (e.g., GPS data, gyroscope,        altimeter, accelerometer, linear acceleration, received signal        strength indicator from nearby emitters such as WiFi access        points, Bluetooth sensors and sensor networks and Cellular        towers);    -   Environmental data (e.g., air quality data, ozone data, weather        data, water-quality data, audible decibel levels, interpreting        measured audio data, measuring luminance lux, interpreting        measured light wavelengths, measuring temperature and gases or        particles—such as formaldehyde (Molecular Formula: H₂CO or        CH₂O); alcohol vapor (Molecular Formula: hydroxyl group-OH,        e.g., IsopropylC₃H₈O or C₃H₇OH, as well as Ethanol: C₂H₆O or        C₂H₅OH); benzene (C₆H₆); Hexane (C₆H₁₄); Liquefied Petroleum Gas        (LPG) which could include a mixture of butane (Molecular        Formula: CH₃CH₂CH₂CH₃ or C₄H₁₀) and isobutene (Molecular        Formula: (CH₃)₂CHCH₃ or C₄H₁₀ or (CHC₄H₁₀)₂CHCH₃); propane        (Molecular Formula: CH₃CH₂CH₃ or C₃H₈); natural coal or town gas        which could include of methane or natural gas (Molecular        Formula: CH₄); carbon dioxide (Molecular Formula: CO₂); hydrogen        (Molecular Formula: H₂); carbon monoxide or possibly smoke        (Molecular Formula: CO); and oxygen (Molecular Formula: O₂) in        the environment surrounding an individual inside and outside the        contextual location of the potential subjects such as home,        office, and including vehicle data—such as speed, location,        amount of time driving, mood while driving, environmental data        in the car).

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. For example, various formsof the flows shown above may be used, with steps re-ordered, added, orremoved.

In the claims, the term “or” is generally intended to be inclusive, notexclusive. For example, the phrase “A or B” should be interpreted asencompassing (1) A only, (2) B only, and (3) A and B together. Thus,absent any modifying word or phrase to specify exclusivity (e.g.,“either A or B” or “only one of A or B”), listed items are not mutuallyexclusive.

Embodiments of the invention and all of the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them. Embodiments ofthe invention can be implemented as one or more computer programproducts, e.g., one or more modules of computer program instructionsencoded on a computer readable medium for execution by, or to controlthe operation of, data processing apparatus. The computer readablemedium can be a machine-readable storage device, a machine-readablestorage substrate, a memory device, a composition of matter effecting amachine-readable propagated signal, or a combination of one or more ofthem. The term “data processing apparatus” encompasses all apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, or multiple processors or computers.The apparatus can include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them. A propagated signal is an artificially generated signal, e.g.,a machine-generated electrical, optical, or electromagnetic signal thatis generated to encode information for transmission to suitable receiverapparatus.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Moreover, a computer can be embedded inanother device, e.g., a tablet computer, a mobile telephone, a personaldigital assistant (PDA), a mobile audio player, a Global PositioningSystem (GPS) receiver, to name just a few. Computer readable mediasuitable for storing computer program instructions and data include allforms of non-volatile memory, media and memory devices, including by wayof example semiconductor memory devices, e.g., EPROM, EEPROM, and flashmemory devices; magnetic disks, e.g., internal hard disks or removabledisks; magneto optical disks; and CD ROM and DVD-ROM disks. Theprocessor and the memory can be supplemented by, or incorporated in,special purpose logic circuitry.

To provide for interaction with a user, embodiments of the invention canbe implemented on a computer having a display device, e.g., a cathoderay tube or LCD (liquid crystal display) monitor, for displayinginformation to the user and a keyboard and a pointing device, e.g., amouse or a trackball, by which the user can provide input to thecomputer. Other kinds of devices can be used to provide for interactionwith a user as well; for example, feedback provided to the user can beany form of sensory feedback, e.g., visual feedback, auditory feedback,or tactile feedback; and input from the user can be received in anyform, including acoustic, speech, or tactile input.

Embodiments of the invention can be implemented in a computing systemthat includes a back end component, e.g., as a data server, or thatincludes a middleware component, e.g., an application server, or thatincludes a front end component, e.g., a client computer having agraphical user interface or a Web browser through which a user caninteract with an implementation of the invention, or any combination ofone or more such back end, middleware, or front end components. Thecomponents of the system can be interconnected by any form or medium ofdigital data communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specifics, these should not beconstrued as limitations on the scope of the invention or of what may beclaimed, but rather as descriptions of features specific to particularembodiments of the invention. Certain features that are described inthis specification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

In each instance where an HTML file is mentioned, other file types orformats may be substituted. For instance, an HTML file may be replacedby an XML, JSON, plain text, or other types of files. Moreover, where atable or hash table is mentioned, other data structures (such asspreadsheets, relational databases, or structured files) may be used.

Particular embodiments of the invention have been described. Otherembodiments are within the scope of the following claims. For example,the steps recited in the claims can be performed in a different orderand still achieve desirable results.

What is claimed is:
 1. A method performed by one or more computers, themethod comprising: training, by the one or more computers, one or moremachine learning models based on collected data indicating (i)activities performed by multiple subjects over time and (ii) performanceresults for a particular activity that the multiple subjects achieved,the one or more machine learning models being trained to be able topredict readiness of subjects to perform the particular activity andachieve a predetermined performance result for the particular activitybased at least in part on prior activities performed by the subjects;accessing, by the one or more computers, a database to obtain statusdata that indicates activities performed by a subject, the status datacomprising data provided from by an electronic device to the one or morecomputers over a communication network; deriving, by the one or morecomputers, a set of feature scores from the status data for the subject,the set of feature scores including values indicative of activitiesperformed by the subject; providing, by the one or more computers, theset of feature scores to the one or more machine learning models thathave been trained to predict readiness of subjects to perform theparticular activity and achieve the predetermined performance result forthe particular activity; generating, by the one or more computers andbased on processing performed using the one or more machine learningmodels and the set of feature scores, a prediction indicating at leastone of (i) a predicted time that the subject will acquire an ability toperform the particular activity and achieve the predeterminedperformance result or (ii) whether the subject will acquire an abilityto perform the particular activity and achieve the predeterminedperformance result; and based on the prediction, providing, by the oneor more computers, output data that is configured to perform at leastone of altering a user interface of a device or altering interaction ofa device with the subject.
 2. The method of claim 1, wherein the one ormore machine learning models comprise at least one of a neural network,a support vector machine, a classifier, a regression model, areinforcement learning model, a clustering model, a decision tree, arandom forest model, a genetic algorithm, a Bayesian model, or aGaussian mixture model.
 3. The method of claim 1, wherein the databasecomprises data generated for the subject over a period of time, thestatus data indicating a sequence of activities that includes activitiesperformed by the subject at different times; and wherein the set offeature scores derived from the status data comprises values indicatingthe sequence of activities performed by the subject at the differenttimes.
 4. The method of claim 1, wherein the set of feature scores isbased on sensor data that is acquired by one or more sensors during oneor more activities of the subject or that indicates one or moreattributes of the subject.
 5. The method of claim 1, wherein the subjectis an individual, and wherein receiving status data comprises receivingat least one of heart rate data for the subject, oxygen saturation datafor the subject, data indicating an exercise distance for the subject,data indicating an exercise intensity for the subject, or dataindicating a duration of an exercise for the subject.
 6. The method ofclaim 1, wherein generating the prediction comprises generating outputindicating a predicted time that the subject will acquire an ability toperform the particular activity and achieve the predeterminedperformance result.
 7. The method of claim 1, wherein generating theprediction comprises generating output indicating whether the subjectwill acquire an ability to perform the particular activity and achievethe predetermined performance result.
 8. The method of claim 7, whereingenerating the output indicating whether the subject will acquire anability to perform the particular activity and achieve the predeterminedperformance result comprises: generating output indicating whether thesubject will acquire an ability to perform the particular activity andachieve the predetermined performance result, given acquisition of theability being subject to one or more constraints that comprise a limiton at least one of: a time to achieve readiness; resources available tothe subject; or a training plan for the subject.
 9. The method of claim1, comprising: generating the collected data by tracking (i) activitiesof each of the multiple subjects over a period of time and (ii) changesin levels of capability of the multiple subjects to perform theparticular activity over the period of time.
 10. The method of claim 1,wherein the subject is a group of individual subjects; and whereinreceiving status data comprises receiving activity data indicatingactivities of individual subjects in the group; and wherein the methodfurther comprises: generating, based on the activity data, a readinessscore for each of the individual subjects in the group, the readinessscores indicating predictions of respective capabilities of theindividual subjects in the group; and generating a group readinessmeasure that indicates a predicted ability of the group of individualsubjects to collectively satisfy one or more group readiness criteria.11. The method of claim 10, wherein providing output comprisesproviding, based on the group readiness measure, output indicating (i) apredicted time that the group will achieve readiness to satisfy the oneor more group readiness criteria or (ii) a likelihood or predictionwhether the group will achieve readiness to satisfy the one or moregroup readiness criteria given one or more constraints.
 12. The methodof claim 10, further comprising: receiving additional activity dataindicating activities of the subjects in the group; generating, based onthe additional activity data, a new readiness score for each of thesubjects in the group; generating a new group readiness measure; andproviding, based on the new group readiness measure, new output dataindicating (i) a new predicted time that the group will achievereadiness to satisfy the one or more group readiness criteria or (ii) anew likelihood or prediction whether the group will achieve readiness tosatisfy the one or more group readiness criteria given one or moreconstraints.
 13. The method of claim 10, comprising: determining thatthe group readiness measure does not satisfy a threshold; and inresponse to determining that the group readiness measure does notsatisfy the threshold, performing at least one of: removing from thegroup one or more subjects determined to have individual readinessscores that are less than an average of individual readiness scores forsubjects in the group; or adding to the group one or more subjectshaving individual readiness scores that are greater than an average ofindividual readiness scores for subjects in the group.
 14. The method ofclaim 1, wherein providing the output data comprises providing arecommendation, determined based on the prediction, of an action toimprove or accelerate acquisition of an ability of the subject toperform the particular activity and achieve the predeterminedperformance result.
 15. The method of claim 1, wherein providing theoutput data comprises providing an indication, for each of one or morecandidate actions, that quantifies a predicted change in readiness ofthe subject, relative to a level of readiness indicated by theprediction, that would be achieved if the candidate action is performedfor the subject.
 16. The method of claim 1, wherein the status dataindicates planned future training exercises for the subject to perform;wherein the set of feature scores includes values indicating the plannedfuture training exercises for the subject to perform; and wherein theprediction is based on the one or more machine learning modelsprocessing feature values indicating (i) past training exercisesperformed by the user, and (ii) the planned future training exercisesfor the user.
 17. The method of claim 1, wherein the collected dataincludes attributes of the multiple subjects, and wherein the trainingincludes training the one or more machine learning models to learnrelationships among different attributes of the subjects and differentlevels of effectiveness of an activity in improving performance ofsubjects that have the different attributes; wherein accessing thedatabase comprises accessing data indicating attributes of the subject;wherein the set of feature scores includes values indicative ofattributes of the subject; wherein the prediction is generated based onprocessing, using the one or more machine learning models, both thevalues indicating activities of the subject and the values indicatingthe attributes of the subject.
 18. The method of claim 1, wherein theset of feature scores includes time series data indicating a sequence ofactivities that the subject performed at different times.
 19. The methodof claim 1, wherein achieving the performance result includesperformance of the particular activity that satisfies a predeterminedthreshold for at least one of speed, latency, accuracy, precision,consistency, efficiency, reliability, scalability, capacity, throughput,volume, or intensity.
 20. The method of claim 1, wherein achieving theperformance result involves completing a task or performing theparticular activity with at least a predetermined level of proficiency.21. A system comprising: one or more computers; and one or morecomputer-readable media storing instructions that, when executed, causethe one or more computers to perform operations comprising: training, bythe one or more computers, one or more machine learning models based oncollected data indicating (i) activities performed by each of multiplesubjects over time and (ii) performance results for a particularactivity that the multiple subjects achieved, the one or more machinelearning models being trained to be able to predict readiness ofsubjects to perform the particular activity and achieve a predeterminedperformance result for the particular activity based at least in part onprior activities performed by the subjects; accessing, by the one ormore computers, a database to obtain status data that indicatesactivities performed by a subject, the status data comprising dataprovided from by an electronic device to the one or more computers overa communication network; deriving, by the one or more computers, a setof feature scores from the status data for the subject, the set offeature scores including values indicative of activities performed bythe subject; providing, by the one or more computers, the set of featurescores to the one or more machine learning models that have been trainedto predict readiness of subjects to perform the particular activity andachieve the predetermined performance result for the particularactivity; generating, by the one or more computers and based onprocessing performed using the one or more machine learning models andthe set of feature scores, a prediction indicating at least one of (i) apredicted time that the subject will acquire an ability to perform theparticular activity and achieve the predetermined performance result or(ii) whether the subject will acquire an ability to perform theparticular activity and achieve the predetermined performance result;and based on the prediction, providing, by the one or more computers,output data that is configured to perform at least one of altering auser interface of a device or altering interaction of a device with thesubject.
 22. One or more non-transitory computer-readable media storinginstructions that are operable, when executed by one or more computers,to cause the one or more computers to perform operations comprising:training, by the one or more computers, one or more machine learningmodels based on collected data indicating (i) activities performed byeach of multiple subjects over time and (ii) performance results for aparticular activity that the multiple subjects achieved, the one or moremachine learning models being trained to be able to predict readiness ofsubjects to perform the particular activity and achieve a predeterminedperformance result for the particular activity based at least in part onprior activities performed by the subjects; accessing, by the one ormore computers, a database to obtain status data that indicatesactivities performed by a subject, the status data comprising dataprovided from by an electronic device to the one or more computers overa communication network; deriving, by the one or more computers, a setof feature scores from the status data for the subject, the set offeature scores including values indicative of activities performed bythe subject; providing, by the one or more computers, the set of featurescores to the one or more machine learning models that have been trainedto predict readiness of subjects to perform the particular activity andachieve the predetermined performance result for the particularactivity; generating, by the one or more computers and based onprocessing performed using the one or more machine learning models andthe set of feature scores, a prediction indicating at least one of (i) apredicted time that the subject will acquire an ability to perform theparticular activity and achieve the predetermined performance result or(ii) whether the subject will acquire an ability to perform theparticular activity and achieve the predetermined performance result;and based on the prediction, providing, by the one or more computers,output data that is configured to perform at least one of altering auser interface of a device or altering interaction of a device with thesubject.